We discuss the control of distributed systems with incomplete data following the notion of no-regret control (or, equivalently, Pareto control) used by Lions in [C. R. Acad. Sci. Paris Ser. I Math., 302 (1986), pp. 223-227] and [C. R. Acad. Sci. Paris Ser. I Math., 302 (1992), pp. 1253-1257]. We associate with the no-regret control a sequence of low-regret controls defined by a quadratic perturbation previously used by Nakoulima, Omrane, and Velin in [C. R. Acad. Sci. Paris Ser. I Math., 330 (2000), pp. 801-806]. In the first part, we prove that the perturbed system corresponds to a sequence of standard control problems and converges to the no-regret (or Pareto) control for which we obtain a singular optimality system. We give also some applications. In the second part, we show how the method can be extended to the evolution case. Equations of parabolic type, Petrowsky type, or hyperbolic type are considered.