In truth, linearity and time-invariance, albeit powerful, are a curse: they are not conducive to an abstract understanding of concepts, tools and ideas and may often be misleading. On the other hand, notions such as manifold invariance, interconnection, coordinates transformations, decomposition, and the principle of optimality facilitate the enhancement of linear, time-invariant, systems theory methods and tools to far more general classes of systems.
We illustrate this perspective by providing abstract and geometric definitions for eigenvalues, poles, moments, Loewner operators and derivative, and persistence of excitation; and by solving interpolation problems, adaptive and robust control problems, and optimal control and game theory problems, for general classes of nonlinear systems. Biography: Alessandro Astolfi was born in Rome, Italy, in He graduated in electrical engineering from the University of Rome in In he was awarded a Ph.
From to he was also an Associate Professor at the Dept. His research interests are focussed on mathematical control theory and control applications, with special emphasis for the problems of discontinuous stabilisation, robust and adaptive control, observer design and model reduction. Distributed systems are the common way to process more data than one computer can store, but they can also be used to increase the pace at which models are trained by splitting the work among many computing nodes.
In this talk, I will study the corresponding problem of minimizing a sum of functions which are respectively accessible by separate nodes in a network. New centralized and decentralized algorithms will be presented, together with their convergence guarantees in deterministic and stochastic convex settings, leading to optimal algorithms for this particular class of distributed optimization problems.
He completed his Ph. He obtained in a Starting Grant and in a Consolidator Grant from the European Research Council, and received the Inria young researcher prize in , the ICML test-of-time award in , as well as the Lagrange prize in continuous optimization in Francis Bach is primarily interested in machine learning, and especially in graphical models, sparse methods, kernel-based learning, large-scale convex optimization, computer vision and signal processing. Domitilla Del Vecchio Massachusetts Institute of Technology , USA Genetic Circuit Engineering Meets Control Theory Abstract: Genetic circuits control every aspect of life and thus the capability to engineer genetic circuits de-novo opens exciting possibilities, from revolutionary drugs and green energy, to bugs that recognize and kill cancer cells.
The remarkable robustness of natural gene networks is the result of million years of evolution and is in sharp contrast with the fragility of synthetic genetic circuits built today. This leads to a daunting design process where each component of a system needs to be redesigned every time a new piece is added. In this talk, I will give an overview of modularity failures in engineered genetic circuits. I will thus introduce a control-theoretic framework, founded on the concept of retroactivity, that addresses the insulation question by mathematically formulating a classical disturbance rejection problem.
Biomolecular feedback control architectures that solve this problem through a form of integral action were used to build two devices in living cells: the load driver and the resource decoupler. These devices aid modularity, thus facilitating predictable composition of genetic circuits to create more sophisticated systems. Control theoretic approaches promise to address many pressing challenges in engineering biology.
Biography: Domitilla Del Vecchio received the Ph. Robert Mahony Australian National University, Australia Equivariant Observers: Robust nonlinear state estimation for robotic systems Abstract: The physical state of a robotic system naturally carries structure; the pose of rigid links can be written as elements of the Special Euclidean group, images taken by a camera of a planar scene can be related by homographies and mapped to elements of the special linear group, etc. Recent work has demonstrated that there is a rich collection of symmetry groups for different robotic problems above and beyond the classical Lie-groups.
Stability is justified in a singular perturbation framework. The efficiency of the control law is validated through simulations demonstrating level curve tracking behaviors of 2-dimensional scalar fields. Keywords: Optimization algorithms , Agents-based systems , Boolean control networks and logic networks Abstract: Convex Mixed-Integer Program MIP has received extensive attention due to its wide applications. We first decouple the feasible region as the intersection of multiple local convex constraints and the relaxed integrality constraints. Then an auxiliary variable vector is assigned to each decoupled constraint with a consensus constraint to keep equivalence.
The updates of the auxiliary variables associated with the convex local constraints as well as the integrality constraint are implemented in a parallel way, followed by the gossip step to drive all auxiliary variable sets into consensus. While this algorithm applies to general convex MIPs, special attention is paid to quadratic and linear constraints in order to obtain a closed-form solution for each subproblem.
While convergence of such DPSA for convex problems has been studied, the presence of the integrality constraint makes it inapplicable. Consequently, convergence of the proposed DPSA and properties at the converging point under certain assumptions is presented. Numerical results on maximum clique problem are provided to sustain the effectiveness of the proposed algorithm. Keywords: Distributed control , Power systems , Predictive control for linear systems Abstract: This paper proposes a centralized and a distributed sub-optimal control strategy to maintain in safe regions the real-time transient frequencies of a given collection of buses, and simultaneously preserve asymptotic stability of the entire network.
In a receding horizon fashion, the centralized control input is obtained by iteratively solving an open-loop optimization aiming to minimize the aggregate control effort over controllers regulated on individual buses with transient frequency and stability constraints. Due to the non-convexity of the optimization, we propose a convexification technique by identifying a reference control input trajectory.
We then extend the centralized control to a distributed scheme, where each subcontroller can only access the state information within a local region. Simulations on a IEEE network illustrate our results. Keywords: Distributed control , Agents-based systems , Adaptive control Abstract: The problem of time-constrained multi-agent task scheduling and control synthesis is addressed. We assume the existence of a high level plan which consists of a sequence of cooperative tasks, each of which is associated with a deadline and several Quality-of-Service levels.
By taking into account the reward and cost of satisfying each task, a novel scheduling problem is formulated and a path synthesis algorithm is proposed. Based on the obtained plan, a distributed hybrid control law is further designed for each agent. Under the condition that only a subset of the agents are aware of the high level plan, it is shown that the proposed controller guarantees the satisfaction of time constraints for each task.
A simulation example is given to verify the theoretical results. Keywords: Chemical process control , Process Control , Optimization Abstract: The efficiency of irrigation systems is critically important for reducing water consumption in agricultural production process, especially with water scarcity nowadays being more and more severe all over the world. Empirical irrigation that often leads to over-watering and results in low yield and water waste should be prevented and substituted by advanced automatic irrigation systems.
In this work, we focus on the data-driven real-time irrigation control and propose a model predictive control MPC -based approach to achieve desired plant root-zone deficit level given variable precipitation and evapotranspiration as disturbance. To take future weather into irrigation decision making, specialized local weather prediction is realized for local irrigation spots where regional weather forecast is less reliable, and the formulation of a dynamic uncertainty set is introduced to account for prediction errors and used in robust MPC design.
The proposed approach is evaluated through a real-world case study in which we demonstrate that the implementation of the data-driven real time irrigation control system effectively facilitates the control of plant root-zone deficit level for local irrigation spots. Keywords: Chemical process control , Process Control , Optimization Abstract: Model predictive control MPC under chance constraints has been a promising solution to complicated control problems subject to uncertain disturbance.
However, traditional approaches either require exact knowledge of probabilistic distributions, or rely on massive multi-scenarios that are generated to represent uncertainties. In this paper, a novel approach is proposed based on actively learning a compact high-density region from available data in form of a polytope.
This is achieved by adopting the support vector clustering, which has been recently utilized in data-driven robust optimization. A new strategy is developed to calibrate the size of the polytope, which provides appropriate probabilistic guarantee. Finally the optimal control problem is cast as a robust optimization problem, which can be efficiently handled by existing numerical solvers.
The proposed method commonly requires less data samples than traditional approaches, and can help reducing the conservatism, thereby enhancing the practicability of model predictive control. The efficacy of the proposed method is verified based on a simulated example.
In a laboratory continuous stirred tank reactor CSTR ran the neutralization reaction of acetic acid and sodium hydroxide. The controlled output was pH value of the reaction mixture and the manipulated variable was the volumetric flow rate of acid. The integral action was designed to remove a steady-state error in set-point tracking.
An uncertain mathematical model of the controlled process was identified from data measured in multiple step responses. Extensive laboratory experimental analysis was performed to tune the weighting matrices of an objective function to optimize the control performance of robust MPC for a laboratory plant. Control performance of the reactor was evaluated using analytical quality criteria. Keywords: Control applications , Adaptive control , Process Control Abstract: The complicated physical and chemical reactions in the smelting process and the blast furnace BF internal complex operating environment have led to the difficulty of establishing the model-based controllers.
Therefore, model free control methods meet the actual needs of the engineering projects. However, due to the sparse characteristic of the molten iron quality MIQ data in BF ironmaking, traditional model free adaptive control based MIQ control methods cannot control such a complex industrial system with strong nonlinear time-varying dynamics. Two groups of verified experiments are performed to evaluate the performance of the controller.
The results show that the proposed method not only has better control performance than the compared traditional CFDL based model free adaptive control method and data-driven model predictive control MPC method, but also can guarantee the bounded-input bounded- output stability of the MIQ output control system for BF ironmaking process.
Keywords: Distributed parameter systems , Estimation , Process Control Abstract: In this paper, we present a model of reservoir pressure dynamics in view of estimating influx during drilling. The distributed nature of the model is shown to have an important impact on the transient behaviour of pressure and flow rate when a liquid influx is present. Then, two observers, designed using a backstepping approach, are used to estimate the distributed reservoir pressure as well as wellbore states.
The relevance of the approach is illustrated in industry-relevant simulations. Keywords: Manufacturing systems and automation , Process Control , Machine learning Abstract: In the manufacturing industry, it is crucial to identify process variables that strongly affect product quality so that high product quality is maintained.
Conventional methods based on variable importance have not necessarily shown good results. In the present work, we propose a new method to estimate variable importance. First, we construct a regression model for predicting product quality from process variables by using support vector regression or gaussian process regression, then we compute variable importance from the sensitivity of the model. It is demonstrated through a numerical example and an industrial case study that the proposed method outperforms conventional methods such as partial least squares and random forest.
Keywords: Predictive control for nonlinear systems , Large-scale systems , Numerical algorithms Abstract: Large-scale nonlinear model predictive control NMPC often relies on real-time solution of optimization problems that are constrained by partial differential equations PDEs. However, the size and complexity of the underlying PDEs present significant computational challenges. In this regard, the development of fast, efficient and scalable PDE-constrained optimization solvers remains central to large-scale NMPC.
As a contribution in this direction, this paper proposes a new efficient preconditioned iterative scheme for optimal control of large-scale time-dependent diffusion-reaction problems with nonlinear reaction kinetics. The scheme combines a custom-made high-order spectral Petrov-Galerkin SPG method with a new preconditioner tailored for the linear-quadratic control problems that underly Sequential Quadratic Programming SQP methods. The preconditioner is matrix-free and amenable to parallelization.
In the absence of control, such processes lead to unstable systems that naturally exhibit finite-time blow-up phenomena. Open-loop simulations demonstrate the ability of the SPG scheme to efficiently control SFI processes, independently of the problem size and the model parameters. We develop a new proof of convexity for the problem that allows the nonlinear dynamics to be modelled as a linear system, then demonstrate the performance of ADMM in comparison with Dynamic Programming DP through simulation.
The results demonstrate up to two orders of magnitude improvement in solution time for comparable accuracy against DP. Keywords: Predictive control for nonlinear systems , Robust control Abstract: Within this paper we consider time optimal robust model predictive control of a robot arm that carries a glass plate. To this end, we propose a respective model and constraints on the strains in the extremal fibers of the glass plate based on the section modulus and tensile strength to avoid breakages. In order to synthesize a control strategy, we propose to use a tailored ellipsoidal tube based model predictive control scheme, which can deal with the highly nonlinear constraints of the glass plate.
The necessity of modeling the strains in the fibers as well as the properties of the proposed robust control method are illustrated in a realistic case study for a KUKA youBot model, which is simulated in the presence of process noise. Keywords: Predictive control for nonlinear systems , Optimization , Uncertain systems Abstract: This paper proposes a computationally efficient algorithm for robust multistage model predictive control MPC.
Read Stochastic Distribution Control System Design: A Convex Optimization Approach
In multistage scenario MPC, the evolution of uncertainty in the prediction horizon is represented via a scenario tree. The resulting large-scale optimization problem can be decomposed into several smaller subproblems where, for example, each subproblem solves a single scenario. Since the different scenarios differ only in the uncertain parameters, the distributed scenario MPC problem can be cast as a parametric nonlinear programming NLP problem.
Instead they can be solved exploiting the parametric nature by a path-following predictor-corrector algorithm that approximates the NLP. This results in a computationally efficient multistage scenario MPC framework. Simulation results show that the sensitivity-based distributed multistage MPC provides a very good approximation of the fully centralized scenario MPC.
Keywords: Predictive control for nonlinear systems , Robust control , Uncertain systems Abstract: This paper presents a robust model predictive controller for discrete-time nonlinear systems, subject to state and input constraints and unknown but bounded input disturbances. The prediction model uses a linearized time-varying version of the original discrete-time system. The proposed optimization problem includes the initial state of the current nominal model of the system as an optimization variable, which allows to guarantee robust exponential stability of a disturbance invariant set for the discrete-time nonlinear system.
From simulations, it is possible to verify the proposed algorithm is real-time capable, since the problem is convex and posed as a quadratic program. Keywords: Predictive control for nonlinear systems , Optimization algorithms , Markov processes Abstract: This paper proposes an algorithm to maximize the power extraction in wind turbine arrays given a varying external wind. Wind turbine arrays, or wind farms, can be viewed as large coupled networks, for which the application of traditional optimization techniques are impractical. In this paper we present an extension to a dynamic programming algorithm previously developed under the condition of uniform wind and extend it to higher-fidelity wind models.
We then update our algorithm for under dynamically evolving wind conditions. Using a Markov chain derived from real-world data, the underlying optimization problem is reformulated in a Model Predictive Control framework. Simulation results are discussed, which demonstrate our algorithm provides improved performance compared to prior results. Keywords: Systems biology , Machine learning , Uncertain systems Abstract: A key objective of systems biology is to understand how the uncertainty in parameter values affects the responses of biochemical networks. Variance-based sensitivity analysis is a powerful approach to address this question.
However, commonly used implementations based on Quasi- Monte Carlo require a very large number of model evaluations, and are thus impractical for computationally expensive models. Here, we present an alternative method for variance-based sensitivity analysis that uses Gaussian process regression. Thanks to the appealing mathematical properties of Gaussian processes, we are able to derive exact analytic formulas for the required sensitivity indices. In this way our approach yields more accurate estimates with significantly less computational cost compared to conventional methods, as we demonstrate for a nonlinear model of a bacterial signaling system.
Keywords: Systems biology , Metabolic systems , Simulation Abstract: Size control is usually exerted in living cells by properly sensing the external inputs like nutrients and, accordingly, by activating the metabolic pathways in order to set and adjust their own growth rate. In this framework, experimental results have recently highlighted the role of metabolic noise, usually neglected because of its averaging over the great deal of reactions involved in metabolic networks.
In this note, a basic model of the interplay among metabolic enzymes activity, resource allocation and growth rate is introduced. A noise source is supposed to affect the enzymatic activity. The model includes a feedforward action of the resource on the enzyme dynamics modulated by growth , as well as a feedback of the enzyme on the resource production rate.
A Stochastic Hybrid System formulation is exploited to investigate how the noise propagates through the metabolic pathway. Model-based results support the hypothesis that fluctuations in the enzyme production perturb cellular growth, and not vice versa, because of an apparent delay in the cross-correlation function.
This result is coherent with single-cell recent experimental results. Keywords: Systems biology , Cellular dynamics , Biological systems Abstract: How living cells employ counting mechanisms to regulate their numbers or density is a long-standing problem in developmental biology that ties directly with organism or tissue size. Diverse cells types have been shown to regulate their numbers via secretion of factors in the extracellular space.
Recommended for you
These factors act as a proxy for the number of cells and function to reduce cellular proliferation rates creating a negative feedback. It is desirable that the production rate of such factors be kept as low as possible to minimize energy costs and detection by predators. Here we formulate a stochastic model of cell proliferation with feedback control via a secreted extracellular factor.
Our results show that while low levels of feedback minimizes random fluctuations in cell numbers around a given set point, high levels of feedback amplify Poisson fluctuations in secreted-factor copy numbers. This trade-off results in an optimal feedback strength, and sets a fundamental limit to noise suppression in cell numbers with short-lived factors providing more efficient noise buffering. We further expand the model to consider external disturbances in key physiological parameters, such as, proliferation and factor synthesis rates.
Intriguingly, while negative feedback effectively mitigates disturbances in the proliferation rate, it amplifies disturbances in the synthesis rate. In summary, these results provide unique insights into the functioning of feedback-based counting mechanisms, and apply to organisms ranging from unicellular prokaryotes and eukaryotes to human cells. Keywords: Systems biology , Biological systems Abstract: Engineering microbial consortia is an important new frontier for synthetic biology given its efficiency in performing complex tasks and endurance to environmental uncertainty.
Most synthetic circuits regulate populational behaviors via cell-to-cell interactions, which are affected by spatially heterogeneous environments. Therefore, it is important to understand the limits on controlling system dynamics and provide a control strategy for engineering consortia under spatial structures. Here, we build a network model for a fractional population control circuit in two-strain consortia, and characterize the cell-to-cell interaction network by topological properties, such as symmetry, locality and connectivity.
Using linear network control theory, we relate the network topology to system output's tracking performance. We analytically and numerically demonstrate that the minimum network control cost for good tracking depends on locality difference between two cell population's spatial distributions and how strongly the controller node contributes to interaction strength.
To realize a robust consortia, we can manipulate the environment to form a strongly connected network. Our results ground the expected cell population dynamics in its spatially organized interaction network, and inspire directions in cooperative control in microbial consortia. Keywords: Systems biology , Cellular dynamics , Metabolic systems Abstract: Recent advances in modeling bacterial cell functioning have deeply renewed questions about bioprocess design and opens the way towards the development of computer aided design CAD for strains.
This article aims at explaining and exploring the consequences opened by the new cell models, investigating the questions related to the biological implementation of optimal strategies and in conclusion the possible role of the complex regulatory network in the retuning of the strain. Understanding and manipulating these computations in nonlinear networks requires a theory of control for abstract objective functions. Real-world applications, such as neural-medicine, constrain which control laws are feasible with less-invasive controllers being preferable.
To this end, we derive a feedback control-law for conic invariance which corresponds to constrained restructuring of the network connections as might occur with pharmacological intervention as opposed to a physically separate control unit. Keywords: Sensor networks , Estimation , Game theory Abstract: In this paper, we consider a nonzero-sum dynamic game that arises in a remote sensing system with a sensor, an encoder, a decoder, and adversarial intervention. At each time step, the sensor makes a measurement on the state of a stochastic process, and then it decides whether to transmit the measurement or not.
If the sensor decides to transmit the measurement, it sends the measurement to the encoder, which then transmits an encoded message to the decoder over an additive noise channel. The decoder generates a real-time estimate on the state of the stochastic process. In this scenario, the cost associated with the remote sensing system consists of a charge for the transmissions made by the sensor, a charge for the encoding power consumed by the encoder, and a charge for the estimation error caused by the decoder.
The components of the remote sensing system have the common objective of minimizing this cost. On the other hand, the additive channel noise is generated by an adversary, which is charged the power of the noise and is rewarded the error of the estimated state. Under some technical assumptions, we obtain a Nash equilibrium solution to this nonzero-sum dynamic game.
Keywords: Game theory , Markov processes , Stochastic optimal control Abstract: We consider an environment where many players need to decide whether to buy a certain product or adopt a trend or not. The product is either good or bad, but its true value is not known to the players. Instead, each player has his own private information on the quality of the product. Each player can observe the previous actions of other players and deduce the quality of the product.
A player can only buy the product once. In contrast to the existing literature on informational cascades, in this work players get more than one opportunity to act. In each turn, a player is chosen uniformly at random from all players and can decide to buy or not to buy. His utility is the total expected discounted reward, and thus myopic strategies may not be best responses. We provide a characterization of structured perfect Bayesian equilibria PBE with non-myopic strategies through a fixed-point equation of dimensionality that grows only polynomially with the number of players.
Based on this characterization we study informational cascades and show that they happen with high probability for a large number of players. Furthermore, only a small portion of the total information in the system is revealed before a cascade occurs. Keywords: Optimization algorithms , Game theory , Optimization Abstract: Submodular maximization is an important problem with many applications in engineering, computer science, economics and social sciences. This algorithm can be distributed among agents, each making local decisions and sharing that decision with other agents.
Recent work has explored how the performance of the distributed algorithm is affected by a degradation in this information sharing. This work introduces the idea of strategy in these networks of agents and shows the value of such an approach in terms of the performance guarantees that it provides. In addition, an optimal strategy that gives such guarantees is identified. Keywords: Transportation networks , Autonomous systems , Fault detection Abstract: Vehicle-to-Infrastructure V2I communications are increasingly supporting highway operations such as electronic toll collection, carpooling, and vehicle platooning.
In this paper we study the incentives of strategic misbehavior by individual vehicles who can exploit the security vulnerabilities in V2I communications and negatively impact the highway operations. We consider a V2I-enabled highway segment facing two classes of vehicles agent populations , each with an authorized access to one server subset of lanes.
Vehicles are strategic in that they can misreport their class type to the system operator and get an unauthorized access to the server dedicated to the other class. This misbehavior causes additional congestion externality on the compliant vehicles, and thus, needs to be deterred. We focus on an environment where the operator is able to inspect the vehicles for misbehavior.
The inspection is costly and successful detection incurs a fine on the misbehaving vehicle. We formulate a signaling game to study the strategic interaction between the vehicle classes and the operator. Our equilibrium analysis provides conditions on the cost parameters that govern the vehicles' incentive to misbehave or not. We also determine the operator's equilibrium inspection strategy. Keywords: Traffic control , Game theory , Transportation networks Abstract: It is known that in traffic systems, less information can lead to better social welfare. This paper studies how to sequentially reveal traffic information to drivers to minimize social cost.
We model this game as a multi-stage Stackelberg game between a designer, who sends public messages about traffic situation to drivers, and drivers, who can help improve the designer's observations. This paper studies the belief systems and the optimal strategies of both players, shows that drivers have a stationary optimal strategy, and provides a recursive formula to compute the designer's optimal strategy. Our simulation results indicate that feedback information from drivers help reduce total social cost and refine their own belief.
In some cases, the designer broadcasts confusing information such that more drivers choose the congested path, which leads to more accurate future observation of the designer. In this way, the designer gains better future social welfare by sacrificing a little current social welfare. Keywords: Optimal control , Cooperative control , Game theory Abstract: The context of graphical games is employed to solve the cooperative control problem for multi-agent systems interacting on graphs.
Together with the need to have faster solution mechanisms urged for new approaches that employ the Dual Heuristic and Action Dependent Dual Heuristic Programming. This class of gradient-based solutions undergoes two main challenges. First, they have to use complex update expressions for the solving gradient-based structures. Second, they may overlook the local neighborhood information, if simpler costate expressions are enforced. A novel approach based on Action Dependent Dual Heuristic Programming is developed to solve the dynamic graphical games and to handle the aforementioned concerns.
This adaptive learning approach is implemented online using means of value iteration and neural networks.
The approximation of the optimal policy does not have priori knowledge about the agents' dynamics, while the value function gradient approximation is shown to depend only on the drift dynamics of the agents. The convergence results of the adaptive learning approach are highlighted by simulation example. Keywords: Optimization algorithms , Networked control systems , Machine learning Abstract: This paper extends our recently proposed distributed optimization algorithm to the time-varying graphs.
The striking feature of the algorithm is that each node only uses binary relative state information from its neighbors. Different from the stochastically time-varying case, the powerful tool of the stochastic approximation theory is no longer applicable here. We show that if the time-varying graphs are uniformly jointly connected, each node of the algorithm asymptotically converges to some common optimal solution of the optimization problem.
We also include simulation examples to validate our results. Keywords: Stochastic optimal control , Robust control , Learning Abstract: In stochastic control applications, typically only an ideal model controlled transition kernel is assumed and the control design is based on the given model, raising the problem of performance loss due to the mismatch between the assumed model and the actual model.
Toward this end, we study continuity properties of discrete-time stochastic control problems with respect to system models i. We study both fully observed and partially observed setups under an infinite horizon discounted expected cost criterion. We show that continuity and robustness cannot be established the under weak convergence of transition kernels in general, but that the expected induced cost is robust under total variation in that it is continuous in the mismatch of transition kernels under convergence in total variation.
By imposing further assumptions on the measurement models and on the kernel itself, we show that the optimal cost can be made continuous under weak convergence of transition kernels as well. Using these continuity properties, we establish convergence results and error bounds due to mismatch that occurs by the application of a control policy which is designed for an incorrectly estimated system model to a true model, thus establishing positive and negative results on robustness.
Compared to the existing literature, we obtain refined robustness results that are applicable even when the incorrect models can be investigated under weak convergence and setwise convergence criteria with respect to a true model , in addition to the total variation criteria. These lead to practically important results on empirical learning in data- driven stochastic control since often, in many applications, system models are learned through training data.
Keywords: Machine learning , Statistical learning , Learning Abstract: We study the finite-sample performance of batch actor-critic algorithm for reinforcement learning with nonlinear function approximations.
Specifically, in the critic step, we estimate the action value function corresponding to the policy of the actor within some parametrized function class, while in the actor step, the policy is updated using the policy gradient estimated based on the critic, so as to minimize the objective function defined as the expected value of discounted cumulative rewards. Under this setting, for the parameter sequence created by the actor steps, we show that the gradient norm of the objective function at any limit point is close to zero up to some fundamental error.
In particular, we show that the error corresponds to the statistical rate of policy evaluation with nonlinear function approximations. For the special class of linear functions and when the number of samples goes to infinity, our result recovers the classical convergence results for the online actor-critic algorithm, which is based on the asymptotic behavior of two-time-scale stochastic approximation.
To improve performance for higher-dimensional problems, thispaper proposes a combination of derivative-free optimization, seeking the global minimizer of a successively-refined surrogate model of the objective function, and an active subspace method,detecting and exploring preferentially the directions of mostvariability of the objective function.
The contribution of otherdirections to the objective function is bounded by a smallconstant. Inverse mapping is used to project data from the active subspace back to full-model for evaluating function values. This task is accomplished by solving a related inequality constrained problem. Test results indicate that the resulting strategy is highly effective on a handful of model optimization problems. Keywords: Machine learning , Learning , Iterative learning control Abstract: Many real-world tasks on practical control systems involve the learning and decision-making of multiple agents, under limited communications and observations.
In this paper, we study the problem of networked multi-agent reinforcement learning MARL , where multiple agents perform reinforcement learning in a common environment, and are able to exchange information via a possibly time-varying communication network. In particular, we focus on a collaborative MARL setting where each agent has individual reward functions, and the objective of all the agents is to maximize the network-wide averaged long-term return. To this end, we propose a fully decentralized actor-critic algorithm that only relies on neighbor-to-neighbor communications among agents.
To promote the use of the algorithm on practical control systems, we focus on the setting with continuous state and action spaces, and adopt the newly proposed expected policy gradient to reduce the variance of the gradient estimate. We provide convergence guarantees for the algorithm when linear function approximation is employed, and corroborate our theoretical results via simulations. Keywords: Agents-based systems , Machine learning , Cooperative control Abstract: In this paper, we study a problem of learning a linear regression model distributively with a network of N interconnected agents in which each agent can deploy an online learning algorithm to adaptively learn the regression model using its private data.
The goal of the problem is to devise a distributed algorithm, under the constraint that each agent can communicate only with its neighbors depicted by a connected communication graph, which enables all N agents converge to the true model, with a performance comparable to that of conventional centralized algorithms. Keywords: Autonomous vehicles , Cooperative control , Automotive control Abstract: The problem of coordinating automated vehicles at intersections can be formulated as an optimal control problem which is inherently difficult to solve, due to its combinatorial nature.
In this paper, we propose a two-stage approximation algorithm based on a previously presented decomposition. We demonstrate the performance of the algorithm through extensive simulation, and show that it greatly outperforms the natural First-Come-First-Served heuristic. The problem consists of finding non-intersecting trajectories of a given number of agents, among a set of admissible curves, to reach a specified configuration, based on minimizing an energy functional that depends on the velocity, covariant acceleration and an artificial potential function used to prevent collision among the agents.
Keywords: Autonomous vehicles , Biologically-inspired methods , Robust adaptive control Abstract: Synchronization is a fundamental function of swarm systems in nature and can be understood as a model of coupled oscillators. Even for manmade complex systems, such as an automated guided vehicles AGVs system in factories, synchronization is also important to enable temporal coordination of AGVs. In the present papter, we propose a new control law for synchronization of coupled relaxation oscillators and apply it to AGV dispatch control.
Our law modulates the threshold of each oscillator, which is a model of a cellular production system. We analyze the stability of the system controlled by the proposed law. It is shown that the phase dynamics of the controlled system can be reduced to those of the Winfree-Kuramoto model.
In addition, on the basis of a stability analysis, we produce a design procedure of the controller and verify the procedure by numerical simulation. Keywords: Autonomous vehicles , Predictive control for linear systems , Distributed control Abstract: Heavy duty vehicle HDV platooning has been widely accepted as a solution to reduce fuel consumption and traffic congestion. However, the control strategy for HDV platoons interacting with other vehicles is not yet well established. This work presents a new framework for handling the requests of passenger vehicles PV plugging in or out of an HDV platoon.
It consists of three main steps. First, the basic cruising control of the platoon is achieved by a distributed model predictive control DMPC scheme. Finally, a transition phase to steady-states guarantees the feasibility of the newly synthesized controllers. Additionally for the plug-in case, we propose a novel approach of Formation Coordinator that determines the optimal location at which the redesigned controller has the best initial feasibility.
The performance of the proposed control framework is illustrated on a multi-vehicle platooning system. Keywords: Autonomous vehicles , Adaptive control , Fault tolerant systems Abstract: In this paper, we develop an adaptive control algorithm for addressing security for a class of networked vehicles that comprise n human-driven vehicles sharing kinematic data and an autonomous vehicle in the aft of the vehicle formation receiving data from the preceding vehicles by wireless vehicle-to-vehicle communication devices.
Specifically, we develop an adaptive controller for mitigating time-invariant, state-dependent adversarial sensor and actuator attacks while guaranteeing uniform ultimate boundedness of the closed-loop networked system. The effectiveness of the proposed approach is demonstrated by an illustrative numerical example involving a platoon of connected vehicles. Keywords: Autonomous vehicles , Robust adaptive control , Lyapunov methods Abstract: This paper presents a geometric adaptive control scheme for a quadrotor unmanned aerial vehicle, where the effects of unknown, unstructured disturbances are mitigated by a multilayer neural network that is adjusted online.
The stability of the proposed controller is analyzed with Lyapunov stability theory on the special Euclidean group, and it is shown that the tracking errors are uniformly ultimately bounded with an ultimate bound that can be abridged arbitrarily. A mathematical model of wind disturbance on the quadrotor dynamics is presented, and it is shown that the proposed adaptive controller is capable of rejecting the effects of wind disturbances successfully. These are illustrated by numerical examples.
Keywords: Networked control systems , Cooperative control Abstract: In this letter, we establish a connection between cooperative control of passivity-short systems, and the regularization of a pair of dual network optimization problems.
- VTLS Chameleon iPortal نتائج البحث?
- Hong Wang | PNNL!
- Radiofrequency heating of plasmas.
We build upon existing works that established, under a passivity requirement, an equivalence between the steady-state behavior of diffusively-coupled network system and the solutions to a pair of convex-dual optimization problems. We show that when the agents are passivity-short, the resulting optimization problems are no-longer convex. By introducing a regularization term to the problem, we then establish that this corresponds to a feedback passivation of each system via an appropriately chosen linear output-feedback gain.
We also obtain conditions on the regularization term such that the resultant closed system possess the so-called maximally equilibrium-independent passivity property and exhibits a solution to their network-level interactions. Finally, we illustrate theoretical results with two case studies. Keywords: Networked control systems , Estimation Abstract: Consider that a remote estimator seeks to estimate the state of a non-collocated discrete-time finite-dimensional linear time-invariant plant that is persistently excited by process noise.
A communication link attempts to relay the state of the plant to the estimator whenever it receives a transmission request. The link experiences packet-drops and has an action-dependent state that is influenced by the history of current and past requests. A controlled Markov chain models this dependence and a given function of the link's state governs the packet-drop probability.
Every randomized stationary transmission policy is specified by a function that determines the probability of a transmission request in terms of the link's state. The article focuses on the design of these policies. Two theorems provide necessary and sufficient conditions for the existence of a randomized stationary policy that stabilizes the estimation error, in the second-moment sense.
They also show that it suffices to search for deterministic stabilizing policies and identify an important case in which the search can be further narrowed to threshold policies. WH is a wireless communication protocol for process automation applications. It is characterised by its multi-hop structure, slotted communication cycles, and simultaneous transmission over different frequencies. We present a solution based on the emulation approach. That is, given an observer designed with a specific stability property in the absence of communication constraints, we implement it over a WH network and we provide sufficient conditions on the latter, to preserve the stability property of the observer.
In particular, we provide explicit bounds on the maximum allowable transmission interval. We assume that the plant dynamics and measurements are affected by noise and we guarantee an input-to-state stability property for the corresponding estimation error system. Keywords: Networked control systems , Lyapunov methods , Stability of hybrid systems Abstract: This paper employs the hybrid-systems-with-memory formalism to investigate transmission intervals and delays that provably stabilize Networked Control Systems NCSs.
We consider nonlinear time-varying plants and controllers with variable discrete and distributed input, output and state delays along with nonconstant discrete and distributed network delays. In other words, we supplant the Lyapunov-Razumikhin conditions and trajectory-based small-gain theorem with linear L2-gains arguments, featured in our previous works, by Lyapunov-Krasovskii functionals to prove UGpAS of interconnected hybrid systems with memory. The selected methodology allows for more general delays e. Our results are applicable to control problems with output feedback and the so-called large delays.
Lastly, a numerical example involving an observer-predictor-based Linear Time-Invariant LTI control system is provided. Keywords: Networked control systems , Hybrid systems , Sensor networks Abstract: Network systems are one of the most active research areas in the engineering community as they feature a paradigm shift from centralized to distributed control and computation. When dealing with network systems, a fundamental challenge is to ensure their functioning even when some of the network nodes do not operate as intended due to faults or attacks.
The objective of this paper is to address the problem of resilient consensus in a context where the nodes have their own clocks, possibly operating in an asynchronous way, and can make updates at arbitrary time instants. The results represent a first step towards the development of resilient event-triggered and self-triggered coordination protocols. Keywords: Networked control systems , Distributed control , Optimal control Abstract: In this paper we consider a finite-horizon optimization problem with a distributed control policy.
CN1673909A - 过程设定控制系统及其控制方法 - Google Patents
The local outputs are sent to a local controller in an intermittent fashion. As a consequence the controller has access to sensor information only if it is sent by the associated local scheduler or by neighboring controllers. We consider generalized event-triggered schedulers which includes time-triggered schedulers as a special case, where time-instants define the events.
This leads to an event-dependent information structure available at each local controller. As a result, the information structure changes, which potentially leads to a non-convex control design problem. For any event-triggered sensing topology, we give a necessary and sufficient condition for convexity of the optimal control problem, by using the quadratic invariance QI property.
Furthermore, we provide an online algorithm that adapts the communication topology among the local controllers and guarantees a step-by-step QI, which translates to a global QI. He realised that the problem could be addressed using interpolation theory from within the general area of mathematics of operator theory. His lecture notes and books were central to the development of the area and collaboration between operator theorists and the control community.
In due course this led to the development of the state-space approach and the two Riccati equation solution presented by Bruce and the present authors. This joint presentation will outline these developments from an historical and mathematical perspective. The idea was to apply a lifting technique to solve two very important optimal control problems.
Francis and collaborators also showed that the lifting technique is applicable in fact to all norm-based optimization problems, in particular to sampled-data quadratic regulation and optimal filtering problems as well as to certain types of structured uncertainty problems including gain margin. Keywords: Stochastic systems , Autonomous systems Abstract: Besides the technical content behind the title, Bruce Francis left a legacy of strong collaborations and cooperation at various levels with a number of colleagues and friends.
On the technical side, the idea of interacting agents became increasingly important at the turn of the century, and Bruce's ideas and work created an important niche. Closely related, is the body of work of Bruce Francis on cyclic pursuit problems that have strong connections to the curve shortening problem in geometric curve evolution theory. In the talk, we will attempt to highlight important ideas and subsequent developments, as well as reminisce on the impact of Bruce' thought and influence to some of us who were his contemporaries. Keywords: Linear systems , Sampled-data control Abstract: Bruce Francis made a first impact on sampled-data systems by initiating the modern approach to sampled-data systems.
He further proceeded to apply the modern method to signal processing. This talk summarizes the crucial difference between the modern and classical designs in terms of some design examples, and also review some basic ideas he pursued in applying the method to signal processing. This formulation was in state space, and thus did not accommodate order uncertainty. In a later paper published in Later, in , Bruce and the speaker derived the internal model in full generality, namely, the case where the plant is perturbed in the graph topology.
This talk will cover the historical evolution of this fundamental result in feedback control theory. Keywords: Distributed control , Agents-based systems Abstract: Based on a combination of consensus and conservation, the paper develops a distributed update for solving linear equations by multi-agent networks, in which each agent only knows just a small part of the overall equation and can only communicate with its nearby neighbors. In the proposed distributed update, each agent knows only two scalar entries of the defining matrix of the overall equation and controls just two scalar states.
Given the underlying networks to be connected and undirected, the proposed distributed update enables agents to collaboratively achieve a solution to the overall equation. Analytical proof is provided for the exponential convergence of the proposed update, which is also validated by numerical simulations. Keywords: Distributed control , Sensor networks Abstract: Monitoring the state of communications in a distributed multilayer network with differing node capabilities requires the maintenance of a backbone which is a connected edge dominating set.
In this paper, we present distributed algorithms that can efficiently create such multilayer resilient connected edge-dominating sets. After establishing the complexity of the problem and our proposed heuristics, we experimentally compare their performance while varying multiple characteristics of the underlying networks. Keywords: Distributed control , Optimal control , Neural networks Abstract: In this paper, continuous and event-sampled approximate optimal distributed control schemes for an interconnected system, with nonlinear subsystem dynamics and strong interconnections, are presented.
The control design problem for the interconnected system is reformulated as an N-player cooperative nonzero-sum differential game wherein the control policy of each subsystem is treated as a player in the game. The Nash solution of this game is used to design the control policy for each subsystem to optimize the performance of the interconnected system.
Approximate dynamic programming ADP , with critic neural networks, is utilized to approximate the solutions of the coupled Hamilton-Jacobi equations, for continuous and event-sampled control implementation. Event-sampling conditions are designed to asynchronously orchestrate the sampling and transmission at each subsystem.
Finally, simulation results are included to substantiate the theoretical claims. Keywords: Distributed control , Cooperative control , Agents-based systems Abstract: This paper addresses a formation control problem of multi-agent systems over generalized relative measurement. Next, we derive a strict class of formation task achievable over the generalized class of measurement data. Then, for a given network graph, an optimal gradient-based distributed and relative controller is designed.
Finally, the effectiveness of the designed controller is illustrated through simulation results. Keywords: Distributed control , Sensor networks , Communication networks Abstract: When a group of agents such as unmanned aerial vehicles are operating in 3-dimensional space, their coordinated action in pursuit of some group objective generally requires all agents to share a common coordinate frame or orientations of the coordinate axes of agents up to an unknown coordinate rotation common to all agents, which are simply referred to as common coordinate axis orientations.
Given coordinate axes that are initially unaligned, this paper considers the process of using direction measurements between agent pairs obtained in their own coordinate frames to achieve orientation localization, i. The process builds on the initial determination of relative orientations of agent pairs in a common coordinate basis. Distributed differential equations then allow determination of a common set of coordinate axis orientations, uniquely up to a common rotation transformation, which can itself be determined if and only if one or more agents have access to global coordinates.
Keywords: Distributed control , Adaptive systems , Filtering Abstract: We study the problem of distributed state estimation in a network of sensing units that can exchange their measurements but the communication between the units is constrained. The units collect noisy, possibly only partial observations of the unknown state; they are assisted by a scheduler which organizes the exchange of measurements between the units. We consider the task of minimizing the total mean-square estimation error of the network while promoting balance between the individual units' performances.