Open Master's Thesis Positions

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate

Nitrogen fertilizer has been critical to supporting high crop yields; however, nitrogen is also susceptible to being washed from agricultural fields into water bodies, where it degrades water quality. The aim of this work is to characterize the risk of a crop nitrogen deficit under different farmer decisions and to identify the potential for reducing fertilizer application without jeoparding crop yields. **The main questions are** **1.** How do we model the dynamics of the nitrogen concentration in the soil? **2.** The evolution of the nitrogen concentration is uncertain due to disturbances (rain), unknown model parameters and imprecise measurements. How can we quantify the risk of crossing a given nitrogen deficit threshold in a meaningful way? **3.** How do we compute an optimal fertilization strategy to minimize the required fertilizer while satisfying the given risk level? **4.** How much (cumulative) fertilizer could be conserved by optimal fertilization? How does shortening the time between measuring soil levels and/or re-applying fertilizer effect the control performance? **References** [1] Paolo D’Odorico et al. “Probabilistic modeling of nitrogen and carbon dynamics in water-limited ecosystems”. In: Ecological Modelling 179.2 (2004), pp. 205–219. [2] Dong Kook Woo and Praveen Kumar. MLCan 2.0. url: https://github.com/HydroComplexity/MLCan2.0. [3] Niklas Schmid et al. “Computing optimal joint chance constrained control policies”. In: arXiv preprint arXiv:2312.10495 (2023). [4] SWAT. SWAT+. url: https://swat.tamu.edu/.

**1.** Study literature on dynamical models of nitrogen [1, 2, 4]. **2.** Implement a simulation based on a realistic model. **3.** Study literature on verification and suitable control tools [3]. **4.** Formulate a suitable risk-constrained optimization problem and solve it using a respective control algorithm. **5.** Analyse the aforementioned questions using the designed simulation and controller.

Please send your resume/CV (including lists of relevant publications/projects) and transcript of records via email to nikschmid@ethz.ch, kwallington@ethz.ch.

Consider a population of self-interested agents, where every agent has a level of karma and urgency (see figure). At every instance, some agents get assigned to a cluster to compete over a partial resource acquisition. The agents bid karma units according to their private states (such as urgency and karma value) to win a portion of the resource. The cluster dimension (number of agents within a cluster) and cluster duration (the time period that a cluster lasts) are random variables, and information on probability distribution and correlation functions are available. The outcomes of the resource allocation will last for the whole cluster duration. Your task is to design a karma mechanism to address fair and efficient proportional resource allocations for the described population.

● Conduct a literature review on dynamic population games, karma games, and game-based approaches for proportional resource allocation. ● Develop a karma mechanism for proportional resource allocation in a population of self-interested agents assigned in random clusters with variable dimensions and durations. ● Investigate the existence of and convergence to the Stationary Nash Equilibrium (SNE). ● Design simulation studies to investigate the performance of the designed karma mechanism. ● Compare the karma mechanism to other policies.

Kimia Chavoshi (kimiac@ethz.ch), Ezzat Elokda (elokdae@ethz.ch)

Online feedback optimization (OFO) is a controller design paradigm for optimizing the steady- state of a dynamical system. OFO uses an optimization algorithm as a feedback controller, and exploits real-time measurements to cope with the uncertainty on the system dynamics and disturbances. On the downside, existing stability guarantees for OFO require the controller to be much slower than the dynamical system; namely, the control gain has to be very small. This “timescale separation” possibly affects transient performance, settling time, and responsiveness to disturbances —especially in problems with high temporal variability. This project investigates conditions under which stability of the closed-loop can be ensured without requiring any timescale separation. To this end, we use the theory of “monotone system”, which applies for instance to freeway traffic models. The student will have the opportunity to learn about OFO and monotone systems, before applying these concepts to a ramp metering problem, both via analytical analysis and numerical evaluation.

1. Literature review on topics in online feedback optimization and monotone systems 2. Become familiar with OFO algorithms and their implementation (MATLAB/Python/Julia) 3. Analysis and numerical evaluation of OFO algorithms for congestion control in a traffic network model 4. Final report and presentation.

We are looking for a talented and highly motivated student with a background in Automatic Control/Applied Mathematics or related fields, optimization and MATLAB/Python/Julia programming. The student must be enrolled in a master program. If you have further questions on this project, please do not hesitate to get in contact with us. If you would like to apply for this project, please email your CV (including lists of projects and optionally any document helpful to evaluate your background) and current grade transcript in PDF to: schandraseka@ethz.ch, mbianch@ethz.ch.

The problem of finding controllers that yield a maximum probability of remaining in a set (invariance), reaching specific states (reachability) or both (reach-avoidance) has a rich history in Dynamic Programming (https://arxiv.org/pdf/2211.07544.pdf). Further, most systems also feature a physical cost objective and trading off the cost against the probability of achieving one of the above specifications yields a problem formulation that has often been considered intractable. While recent developments propose a computationally effective approach, they require full information of the stochastic model dynamics (https://arxiv.org/pdf/2312.10495v1.pdf, https://arxiv.org/pdf/2402.19360v1.pdf). This project aims to alleviate this restriction using learning-based approaches, e.g., Reinforcement Learning. The ball-on-a-plate system, a mechanically actuated plate that balances a ball in its middle, will act as a benchmark. The overall goal is to manouvre the ball to a sequence of sets as fast as possible while retaining a high enough reliability. Recently equipped with a simple-to-use python interface the system allows for an easy deployment of control algorithms.

The goal is to 1. Design learning-based algorithms to trade off invariance, reachability and reach-avoid specifications against cost objectives. 2. Implement and validate the algorithm on the Ball-on-a-Plate system, including creating a good visual presentation of the approach.

Please send your resume/CV (including lists of relevant publications/projects) and transcript of records via email to nikschmid@ethz.ch, mfochesato@ethz.ch. A basic familiarity with reinforcement learning or dynamic programming is expected.

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate

Global decarbonization targets and energy demands are driving a rapid integration of distributed renewable energy resources and efficient conversion and storage technologies in energy systems. Distributed energy hubs transforms the system from a conventional grid-centric to a decentralized prosumer-centric model. The optimal operation of energy hubs and peer-to-peer trading increases energy efficiency, flexibility, and eases integration of renewable resources ~\cite{behrunani-reviews}. These changes in the grid topology has also made peak shaving crucial for the future of the system. Peak shaving is a term used in energy management to describe reducing the energy consumed during peak demand on the electric grid. Peak demand is a period when energy consumers use the most amount of electricity. Leveling out peaks in electricity use not only reduces the stress on the network but also reduces costs the highest peak of the month is charged at a very high tariff, hence reducing peak electricity demand is an effective strategy to reduce electricity expenses. In this project the goal is to design a strategy to detect the peaks in real time before they occur and formulate a controller to deploy flexible energy hub resources such as battery energy storage or thermal storage along with peer to peer electricity and thermal trading and thermal flexibility of the connected buildings to mitigate these peaks before they occur thereby reducing the peak demand and consequently the energy costs.

The tasks that will be carried out in this thesis are (tentative): 1. Literature review: study the existing strategies for peak reduction in networks 2. Design a suitable real time peak detection strategy that relies on past data and environmental information to detect oncoming peaks in order to deploy flexible capacities. 3. Formulate a controller to deploy flexibility strategies. 4. Analysis by simulating different load scenarios and comparison to other peak reducing strategies. 5. Documentation and final presentation of the work

Varsha N. Behrunani bvarsha@ethz.ch

A Bregman divergence is a measure of the difference between two vectors; the simplest example is the squared Euclidean distance. Bregman divergences offer a more general and flexible alternative to quadratic functions when it comes to Lyapunov functions for nonlinear systems. The goal of this project is to test a new approach for constructing Bregman Lyapunov functions. This method has some key advantages: it does not require knowing the system’s equilibrium point (making it ideal for analyzing the stability of optimization algorithms), and it does not rely on restrictive ``contractivity'' assumptions. This flexibility comes at the cost of solving a complex optimization problem. To tackle this issue, we can explore one or more of the following computational strategies: (i) sum-of-squares (SOS) optimization, (ii) neural network approximations, or (iii) gridding of the state space. The student will gain hands-on experience with advanced topics such as SOS optimization and computer-aided Lyapunov analysis. By testing and refining the method, the student will have the opportunity to contribute to the development of novel tools for stability analysis of nonlinear systems. For those looking to push the boundaries of this research, there is also the possibility to explore applications in control synthesis.

mbianch@ethz.ch

The need to ensure the efficient operation of dynamical systems of increasing complexity (e.g., interconnected power grids) has recently sparked renewed interest in “direct data-driven control” methods. This term refers to designing controllers using exclusively measured data, without the identification of a model –the most expensive and time-consuming process. Since the true plant is unknown, the goal consists in finding a robust controller that can work for all the plausible systems – i.e., all those compatible with the available measurements. This project applies the paradigm of direct-data driven control to the operation of electric microgrids. We model a microgrid as a Markov Jump Linear System (MJLS), where the dynamics jump between different modes. This approach has proven useful in the literature, as it results in computationally cheaper design problems than using complex nonlinear models. Our goal is to design switching controllers (when the system jumps, the controller also changes) that ensure stability, by solving data-based linear matrix inequalities. The student will learn about linear matrix inequalities and data-driven control, before designing and testing numerically controllers for a standard power system model.

mbianch@ethz.ch

The integration of renewable energy sources and distributed generation in modern power systems has increased significantly. This resulted in an increased use of power-electronics-based converters, changing the dynamical behaviour of the grid and raising the need for accurate grid impedance estimation methods. Knowledge of grid impedance at the point of common coupling is essential for online stability assessment of grid-connected converters [1]. It is also used for optimizing the closed-loop performance via, e.g., adaptive control design. However, due to the grid's dynamic and nonlinear nature, impedance estimation becomes particularly challenging, in particular when conducted in real-time under closed-loop operation (see e.g., [2]). A new non-parametric identification method has shown potential for accurate and rapid grid impedance identification. While initial simulations have demonstrated the potential of the method, experimental validation using hardware is crucial to evaluate the method's performance under real-world conditions. This project aims to use two converter emulators ([3]) available in the Automatic Control Laboratory to experimentally validate the new approach. The main goals are to replicate realistic converter/grid conditions, assess the accuracy and robustness of the method, and to explore the limitations and performance boundaries of the method. References: [1] J. Sun, "Impedance-Based Stability Criterion for Grid-Connected Inverters," in IEEE Transactions on Power Electronics, vol. 26, no. 11, Nov. 2011 [2] V. Haberle, et.al., "MIMO Grid Impedance Identification of Three-Phase Power Systems: Parametric vs. Nonparametric Approaches," 2023 62nd IEEE CDC [3] https://imperix.com/products/power-electronics-test-bench/ **Prerequisites** We are looking for a motivated student with a background in power systems/power electronics, and a previous exposure to system identification or signal processing. Knowledge of grid-connected converters and impedance estimation methods would be highly beneficial. Familiarity with experimental configurations and lab instrumentation is desirable. Competitive candidates will have prior research or work experience that demonstrate the ability to learn and problem-solve independently.

1. _Literature Review_: Review state-of-the-art literature on non-parameteric and parameteric grid impedance identification techniques, and learn the theoretical basis of the LPM method. 2. _Set up an experimental configuration_: Use the available imperix power electronics test benches to replicate a grid-connected converter scenario, allowing for safe and accurate data collection for impedance estimation. 3. _Data collection and analysis_: Collect experimental data under a variety of grid and excitation conditions. Analyze the accuracy and robustness of the LPM method, and compare to alternative estimation methods. 4. _Results interpretation and conclusions_: Document the findings, noting any discrepancies between simulation-based results and experimental outcomes. Propose potential improvements to the estimation method or additional experiments for further study, with the aim of ensuring reliability and robustness in real-world applications. 5. _Benchmark comparisons_: Examine the performance of alternative methods in a benchmark comparison study, highlighting the differences in terms of accuracy, used devices and experimentation time. 6. _Reporting_: Prepare a final report and a 30 minutes final presentation.

Please send your CV and transcript of records via email to Mohamed Abdalmoaty mabdalmoaty@ethz.ch and Xiuqiang He xiuqhe@ethz.ch.

The use of renewable energy sources and distributed generation, such as wind turbines and solar power plants, in modern power systems has been increasing significantly. This resulted in a heightened use of power-electronics-based converters, changing the dynamical behaviour of the power grid and raising the need for accurate grid impedance estimation methods. Knowledge of grid impedance at the point of common coupling is essential for online stability assessment of grid-connected converters [1]. However, due to the grid's dynamic and nonlinear nature, impedance estimation is particularly challenging (see e.g., [2]). The grid-connected converter's controller must maintain stable operation while simultaneously allowing the injection of small excitation signals necessary for grid impedance estimation. This creates a fundamental trade-off: while a good controller and converter-grid interface filter improve system stability and disturbance rejection, they can also suppress the grid's response to the excitation, reducing the estimation accuracy. On the other hand, injecting larger excitation signals could improve the quality of estimation but risks triggering the converter protection system and/or exciting nonlinear grid behaviors. Moreover, it is not clear what the best class of excitation signals is nor the best injection point within the converter controller. This project aims to address this trade-off and questions by developing optimal excitation schemes for grid impedance estimation using grid-connected converters. A main goal is to obtain a good signal-to-noise ratio at the frequencies of interest while respecting experimental time and signal magnitude constraints. A class of excitation signals of potential interest is random phase multisines [3]. Recent related results in the literature are [4,5] which are based on modelling the noise and the use of multiple sinusoidal injections. References: [1] J. Sun, "Impedance-Based Stability Criterion for Grid-Connected Inverters," in IEEE Transactions on Power Electronics, vol. 26, no. 11, Nov. 2011 [2] V. Haberle, et.al., "MIMO Grid Impedance Identification of Three-Phase Power Systems: Parametric vs. Nonparametric Approaches," 2023 62nd IEEE CDC [3] Godfrey, Keith, ed. Perturbation signals for system identification. Prentice Hall International, 1993. [4] Y. Zhu, et.al, "Injection Amplitude Guidance for Impedance Measurement in Power Systems," in IEEE Transactions on Power Electronics, vol. 38, no. 6, June 2023 [5] H. Li, et.al., "A Noise Interference Suppression Method With Optimized Frequency Adjustment for Impedance Measurement," in IEEE Transactions on Industrial Electronics, vol. 71, no. 12, Dec. 2024 **Prerequisits** We are looking for a motivated student with excellent written and verbal communication skills, and the ability to learn and problem-solve independently. Competitive candidate will have a background in systems & control, signal processing, and power systems/electronics. A prior experience or coursework in system identification and/or grid-connected converters would be highly beneficial. Proficiency in MATLAB/Simulink or a similar simulation environment is required for implementing and testing the methods.

1. _Literature review & problem definition_: Conduct a literature review on existing grid-impedance estimation techniques, as well as excitation design schemes. Define the requirements and limitations for accurately estimating grid impedance in a real-time, closed-loop setting, while addressing factors that impact estimation accuracy under practical conditions (e.g., experiment time, amplitude of excitation signals, etc). 2. _Development of optimal excitation framework_: Formulate an "optimal" excitation framework tailored for grid impedance estimation via grid-connected converters. 3. _Excitation signal design_: Generate and test the excitation signal in simulation, highlighting the impact on estimation accuracy. Explore the potential of iteratively refining the signal design to improve accuracy. 4. _Benchmark comparisons in simulation_: Compare the performance of your method(s) to existing ones as well as those using arbitrary excitation on benchmark models. Analyze the results to determine the practical effectiveness of the proposed approach for real-time applications in power systems. 5. _Reporting_: Prepare a final report and a 30 minutes final presentation.

Please send your CV and transcript of records via email to Mohamed Abdalmoaty mabdalmoaty@ethz.ch and Xiuqiang He xiuqhe@ethz.ch.

**Motivation.** Neural networks (NNs) achieve outstanding empirical performance when approximating arbitrarily complex functions. Yet, certain prediction tasks require the output of the NN to satisfy hard specifications, a goal that standard architectures typically struggle with. One such important task is approximating the solution of a constrained optimization problem, which is especially useful when: 1. the problem in question is difficult (e.g., non-convex, mixed-integer); 2. the problem is easy but has to be re-solved at a high frequency (e.g., Model Predictive Control). Although standard NNs can be used to computed a suboptimal solution for such problems, constraint satisfaction is not guaranteed, severely limiting their applicability. **Approach.** Our main tool to address these issues will be hard-constrained neural networks (HCNNs) that, essentially, consist of any standard NN architecture followed by a "projection layer". This final projection layer ensures that the HCNN output satisfies certain constraints. There are various ways to achieve this, such as: (a) orthogonal projection; (b) ray projection; (c) soft-constrained formulations. The main challenge that projection layers introduce is that backpropagation cannot be done in closed-form (as with standard layers). Therefore, the method that different projection layers use to implement backpropagation significantly differentiates their predictive and computational performance.

Our goal in this project is to benchmark different HCNN architectures, identify the most promising ones and deploy them to solve challenging applications. Candidate applications include mixed-integer programs and online path planning (but this can be adapted to the student's interests). Through this project, the student will gain both strong coding experience and insights on convex optimization. Particularly, HCNNs fall under the very actively-researched fields of NNs with implicit layers and differentiable optimization.

Please send your resume/CV (including lists of relevant publications/projects) and transcript of records in PDF format via email to {pgrontas,aterpin,ebalta}@ethz.ch.

Hydroponics is an agricultural technique of rising interest in research and with first industrial successes to produce herbs. Not only does this technique require less water than conventional agriculture, but it may also find applications in space exploration to grow crops in space stations and on Mars. Yet, hydroponics comes at the cost of high energy and intense initial resource requirements. This project in particular focuses on the former. We aim to grow requested amounts of biomass with as little as possible energy investments (light, heating, water circulation…). The project can be divided into the following steps.

1. Select a suitable model from the literature to build a simulator for the growth of crops. 2. Implement a suitable controller that minimizes energy investments while guaranteeing a desired biomass production. 3. If time permits: Design model-free approaches to leverage data from experiments to either learn a model or an optimal controller A background in dynamic programming and / or reinforcement learning may be useful.

Please send your resume/CV (including lists of relevant publications/projects) and transcript of records via email to nikschmid@ethz.ch, kwallington@ethz.ch.

Model predictive control (MPC) is ubiquitus in industry and academia. The core idea of this control technique is to utilize a model of the process dynamics to construct a prediction of how the system will evolve within a certain prediction horizon, starting from a given initial state. Next, the MPC dynamically determines the optimal control actions within the horizon by solving an optimization problem with the goal of minimizing some cost function (for example steering the state of the system to the origin or tracking a reference). When deployed on a real system, the optimization is repeated at each time-step, and the control sequence is applied only for the first portion of the prediction horizon. As time progresses, the optimization process is repeated, and the horizon "recedes" forward in time. This constant re-optimization ensures that the controller actively responds to variations of the environment, thus enabling feedback. The prediction that the MPC makes at every timestep is referred to as "open-loop" control, since it doesn't take into account future measurements beyond the current time. The main source of suboptimality in MPC is the discrepancy between the open-loop prediction and the true trajectory (closed-loop) obtained with the receding horizon paradigm. This is because the objective function in the MPC problem involves the open-loop, rather than the closed-loop. One way to reduce the suboptimality is to increase the length of the prediction horizon. In this way the MPC is able to capture the system's evolution over a longer period, thereby making choices that better discount between short and long term benefits. However, a longer prediction horizon requires more computational resources and this may not be possible in applications where the MPC is expected to run at fast rates. Another possibility to improve the closed-loop performance of MPC is to modify the cost function of the MPC problem in such a way that the control action is chosen optimally for the closed-loop. We can obtain the optimal cost function by solving a policy optimization problem. These problems, generally addressed in the field of reinforcement learning, involve adjusting the parameters of the policy (in this case, the MPC controller) to improve its performance. Reinforcement learning algorithms can be implemented using gradient-based methods. These methods involve computing the gradient of some performance measure with respect to the MPC parameters and then updating the parameters in the direction that improves performance. To obtain the gradient of the entire closed-loop trajectory with respect to the design parameters, we can utilize the approach proposed in [1], which utilizes a backpropagation scheme. The same gradient-based techniques can also be used to ensure that the MPC scheme is robust against unmodeled dynamics and / or stochastic noise [2] (see pdf for a numerical example).

In this project, we would like to develop a solution to the problem of closed-loop design of MPC by following the initial efforts in [1]. The goals are the following: - improve the closed-loop performance of MPC in the context of partially known or fully unknown systems subject to stochastic disturbances; - consider a range of operating conditions; - compare the performance against state of the art reinforcement learning algorithms. The project can be tailored to focus more on the theoretical aspects, or on the implementation and testing of the algorithms in simulation.

We are looking for motivated students with some prior knowledge about model predictive control and either Python (preferably) or Matlab. Knowledge about numerical optimization is a plus. Please send your resume/CV (including lists of relevant publications/projects) and transcript of records in PDF format via email to rzuliani@ethz.ch and efe.balta@inspire.ch. Otherwise, come meet us at the IfA Open House at the industry, control and robotics panel, you can find information about the event in our homepage at https://control.ee.ethz.ch

Please, see attached file.

Through this project, the student will gain insights into advanced game-theoretic concepts, focusing on the issues of Stackelberg games, statistical learning and learning in games. The student will develop skills in working with mathematical models of games and optimization problems, data analysis, as well as translating theoretical findings in a well-written report/paper. For more detailed description, please, read attached file.

Dr. Ilia Shilov: ishilov@ethz.ch

Designing suitable controllers for building energy systems presents considerable challenges due to the variability in system characteristics from one building to another and the seasonal variation in disturbances. Additionally, the inherent nonlinearities of HVAC system dynamics complicate controller design. Current HVAC controller tuning processes are highly time-consuming, and controllers often need recalibration following building retrofits. Developing adaptive controllers that can autonomously adjust to specific HVAC systems and respond to configurational and seasonal changes is thus of high interest. In pursuit of these objectives, this project will: - explore control strategies from the domain of data-driven control like Data-enabled Predictive Control (DeePC) and Reinforcement Learning (RL). - examine potential architectures, such as Hierarchical RL (HRL), to optimally implement data-driven controllers for HVAC applications. - assess the performance of the proposed approach in a representative building control scenario. The student can utilize several existing building and component models. Familiarization with these resources and integration of the developed code in the existing framework is expected. The project will primarily involve simulation work, with no experimental activities planned. Case studies based on real-world data may also be explored.

The primary goals of the project include: - Investigate and evaluate control methods and architectures from the field of data-driven control. - Implement and assess these methods for building energy system control within a simulation environment. The findings from this project will potentially contribute to research publications from the laboratory. If results are particularly promising, they may serve as the foundation for a stand-alone publication.

We seek a highly motivated student with a background in control. Proficiency in numerical methods, optimization, thermal system modeling, building control, and familiarity with data-driven techniques or machine learning are advantageous. - Specific experience with building systems is not required. - Strong foundation in control theory, ideally also for nonlinear control systems. - Strong interest in applying and adapting/developing data-driven control methods. - Proficiency in Python and/or Matlab programming languages. - Knowledge of modeling and simulating dynamic systems using Matlab/Simulink is a plus. - Excellent proficiency in English and strong communication skills are essential. - Depending on the project’s focus, the student will collaborate with Belimo Automation’s research group and may have limited access to necessary company resources for development and testing. Please send your resume/CV (including relevant publications/projects) and transcript of records in PDF format to efe.balta@inspire.ch and bspoek@ethz.ch.

Please, see attached file.

The student will gain insights into advanced game-theoretic concepts, focusing on the issues of Stackelberg games, imperfect information, credible threats, and bounded rationality. The student will develop skills in developing mathematical models of games and optimization problems, simulating games, as well as translate theoretical findings into well-written report/paper. For more detailed description, please, see the file attached.

Dr. Ilia Shilov: ishilov@ethz.ch

Modern control methods often rely on explicit online computation. That is, for every time step, the controller needs to perform some optimization, numerical algebra or integration in order to determine the inputs. Naturally, such methods give rise a trade-off: For the same implementation more accuracy requires more computation time. Given the extremely high actuation frequency of modern control plants, and the increasing scale of, for instance, online optimization problems, the analysis and acceleration of such implementations is of great interest. Generally, performance analysis of such controllers relies on time scale separation. In simple terms, we assume that ‘enough’ computation can be performed to obtain a control signal that is ‘close’ to optimal. However, this leaves a number of important question unanswered: ˆ Beyond the analysis of closed loop stability, can we derive or minimize bounds on the transient behavior? ˆ Can we characterize precisely which algorithms in a specific class yield closed-loop stability? ˆ Given the previous, can we design algorithms that do not just converge quickly, but yield robustly stable closed loops? In order to understand these question, and more generally the behavior of closed loops between numerical methods and dynamical systems, this project approaches the algorithm as a dynamical system itself. In doing so, the usual language of convergence of algorithms can be viewed as a special case of stability theory. Moreover, beyond the stability analysis, this opens up the use of controltheoretical methods to analyze for instance noise rejection and, more generally, closed-loops between plants and algorithms.

The student will: 1. learn and understand concepts related to the topic, including incremental dissipativity and numerical analysis; 2. explore and review the relevant literature; 3. perform simulations on closed loop systems to validate and compare different methods; 4. bring together the theory and analyze the results;

Dr. Jaap Eising, jeising@ethz.ch.

The thesis should: - Prepare electricity demand profiles for different building types in the district - Prepare techno-economic data on energy conversion and storage technologies - Techno-economic assessment of electricity storage solutions (Li-ion, redox-flow, etc.) - Techno-economic evaluation of storage configurations (centralized vs. decentralized). - Assess the role of different energy storage technologies and configurations in enhancing grid flexibility and optimizing energy flows in the district.

This thesis aims to perform a (pre-) feasibility study on various electricity storage solutions that can be implemented within the district.

Dr. Binod Koirala (binod.koirala@empa.ch) Group Leader Buildings & Cities, Urban Energy Systems Lab

The key steps envisioned to complete the project are as follows: 1. Literature review - conduct a literature review on the modeling of BTES systems and district heating and cooling networks, with particular emphasis on the modelica language (4 weeks). 2. Model development - Develop a detailed Modelica-based model of the high-temperature BTES system of the Empa campus, including interactions with the district heating and cooling networks (6 weeks). 3. System verification - Use the developed model to verify the current system design and identify areas for improvement (6 weeks). 4. Design improvement suggestions - Analyze the results and propose potential design modifications (4 weeks). 5. Documentation and report writing - Create detailed documentation of the model and writing of the final report (4 weeks).

This project aims to develop high-fidelity models of the high-temperature borehole thermal energy storage system, focusing on its integration with the district heating networks of the Empa campus. The model will be implemented using the Modelica language within the open-access OpenModelica software. The developed model will be documented in detail to ensure future usability and maintainability. The model will also contribute to the ongoing digital twin efforts of the Empa campus, allowing for real-time verification and optimization of system performance. The following key objectives will be addressed: • Verification of system design: Assess the current design of the HT-BTES system and identify any discrepancies between the design assumptions and real-world operation; • Suggest design and operational improvements: Propose modifications or enhancements to the current system design and operational strategies to improve its long-term sustainability and integration within the district network.

gabriele.humbert@empa.ch

**Key Responsibilities:** - Assist in the assembly of the LED Solar Simulator, including: - Assembly of solar simulator parts (including isolators and reflectors). - Adjusting and securing components on the LED chips - Installing and positioning various parts, including magnets and reflectors, to complete the reconstruction of the simulator. - Work closely with our team to ensure the successful reassembly of the simulator. **Working Hours:** 48 hours in total, spread over two weeks (3 days per week, 8 hours per day). **Required Skills:** - Interest in mechanical assembly and hands-on tasks. - Willingness to learn and work independently after initial training. - Preferably an ETH student with vocational training. - Interest in physics or lighting technology is a plus. - Handy with tools and basic assembly work. **Required Documents:** - CV (emphasizing any relevant job experience) - A motivation letter, a maximum of one page, explaining your background and why you think you are suitable for this role. (optional)

As part of a short-term maintenance project (2 weeks), we are seeking a motivated student assistant to help reconstruct our state-of-the-art LED Solar Simulator (artificial sun).

Dr. Sarah Crosby (crosby@arch.ethz.ch)

The envisioned steps for the project are: 1. Literature review: Explore relevant literature on hydrogen technologies, waste heat recovery, and their application in small-scale districts and energy communities. 2. Scenarios definition: Based on-going projects at Empa, specific case studies and scenarios will be defined. 3. Model Development: Extend existing models, such as those in the EhubX optimization tool [4], to incorporate and calibrate hydrogen storage, fuel cells, electrolyzes and waste heat recovery. 4. Numerical Simulations: Conduct sensitivity analyses to evaluate the performance of different configurations and scenarios. 5. Analysis and Reporting: Analyze results and provide recommendations on how to integrate hydrogen technologies and waste heat recovery to achieve decarbonization targets

The primary goal of this thesis is to develop and test models for the integration of hydrogen technologies and waste heat recovery in prosumer energy communities. The research will address the following key questions: • What is the optimal configuration for integrating hydrogen technologies in energy communities? • In which scenarios do hydrogen technologies provide the most significant benefits in terms of cost reduction and carbon savings?

gabriele.humbert@empa.ch

Buildings in Switzerland contribute significantly to energy consumption, accounting for 42% of total energy use and 26% of CO2 emissions, with heating representing 68% of this energy. Addressing the challenge of reduc-ing heating energy consumption is critical, though it must be done while maintaining tenant comfort. One approach to this is the implementation of an adaptive system. Our adaptive system is based on optimiz-ing heating curves using Contextual Bayesian Optimization. Heating curves are general function that describes the relationship between ambient temperatures and supplied heating power to secure a pleasant environment for the tenants. The simplest heating curves are 2 point linear heating curves described by two parameters. The optimization process involves adjusting the underlying heating parameters, we call the process of finding new parameters short action, through simulation, measuring energy and comfort costs to minimize energy use while ensuring comfort and update the heating curve with the additional information, before starting the next iteration. In our current setting we optimize a 2 point linear heating curve limit to 1 action and 1 contextual information. A key advantage of our approach is its simplicity, that makes it easy to set-up and choose parameters for and representing the benefits in a way that is understandable for the target audience (see additional information attached). However, while adaptive systems offer substantial improvements over static models, their complexi-ty and black-box nature can pose challenges in understanding, which may hinder broader adoption. Ensuring the model's transparency and usability is essential for its success. For more details see the attachment AHA: Contextual Bayesian Optimization of Heating Curves. This semester thesis will build on the existing system and take a closer look at the following research ques-tions: Heating Curve Parameterization We want to answer is which parametrization of the heating curve could be most beneficial. Some simple ex-amples of curves that could be used are: linear curve, segmented linear curve, quadratic interpolation and clipped quadratic interpolation. A more complex curve (for example one with multiple inflection points) could also be considered. Interpretability and Interactivity To maximize the model's benefits, it must not only achieve the stated objective of reducing cost while maintaining comfort, but also surpass the difficult hurdle of representing the benefits in a way that is understandable for the target audience. The plan is to create a simple interface showing the acquired data-points, along with what the model estimates is: - the "safe" region of the parameter space based on constraints imposed by the maximum tolerated dis-comfort and physical limitations; - the mean of the cost function, showing the quantity explained by the comfort level and the one explained by the energy cost; - confidence bound of the cost function. The breakdown of the cost function is the most significant part in this interface, as it could show a trade-off between the achieved comfort levels and the energy cost. The safe region is also an essential element as it displays the trade-off between exploration and exploitation in the model: a narrow safety region may suggest that the model is being too conservative, leading to subop-timal results. To get a clearer idea of the model behavior, the interface will include an interactive feature, allowing users to drag the various points of the heating curve. New values for safety regions and cost functions will then be computed based on the updated interpolated curve. The interface will offer two kinds of visualizations of the heating curve: one displaying the cost function esti-mates for every point, and another that focuses on one of the two boundary points. The second visualization will show the two-dimensional cost function level sets and allows dragging the endpoints in any direction, which might be too unintuitive for the first visualization. Choice of Algorithm The final research question revolves around choosing the algorithm, particularly focusing on two aspects: in-corporation of context and safety considerations. 1. Regarding incorporation of context, following the suggestions in [1], we aim to understand how integrating contextual information influences the model. In particular we consider the following alternatives: - Ignore the context; - Divide the data into buckets of equal width; - Use a composite kernel to combine context and parameters. 2. Regarding Safety considerations in the acquisition function, different algorithms offer varying guarantees in how they approach safety while trading-off exploration and exploitation. Exploring multiple approaches could lead to an algorithm that optimizes effectively while ensuring safety. The main options under consideration in-clude: - SafeOPT [2], an approach that ensures that every sampled point is safe with high probability. At each iteration, it selects a point that could expand the safe set (expander) or one that could improve the upper confidence bound (maximizer); - Safe GP-LCB as proposed in [3], this approach extends the GP Lower Confidence Bound by including a safe region, this could lead to faster convergence; - Including a large penalty for points which are likely to be unsafe. The algorithms will be compared on several criteria: - convergence speed; - optimality; - cumulative violations regret; - cumulative cost regret. Understanding the effects of various forms of non-linearities on the mentioned criteria is also crucial - [1] Krause, Andreas and Ong, Cheng Soon (2011). Contextual Gaussian process bandit optimization, https://dl.acm.org/doi/10.5555/2986459.2986732. - [2] Sui, Yanan and Gotovos, Alkis and Burdick, Joel and Krause, Andreas (2015). Safe Exploration for Optimiza-tion with Gaussian Processes, https://proceedings.mlr.press/v37/sui15.html. - [3] Marcello, Fiducioso and Sebastian, Curi and Benedikt, Schumacher and Markus, Gwerder and Andreas, Krause (2019). Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning, https://arxiv.org/abs/1906.12086.

**Deliverables:** - Literature Review und study of software tools - Parametrization of Heating Curves: Implementations of various heating curve models (linear, quadratic, etc.), Documentation comparing the performance of these models under different conditions, Analysis of the pros and cons of complex vs. simple heating curve - Development of a prototype of a UI that visualizes: Safe regions of the parameter space. Cost function components (comfort level vs. energy cost). Confidence bounds of the cost function. Interactive feature allowing users to modify the heating curve and observe changes in the safety region and cost function. - Algorithm Selection and Implementation: Implementation of various algorithms (SafeOPT, Safe GP-LCB, etc.) to ensure safe and efficient ex-ploration of the parameter space. - Comparison of algorithms based on: Convergence speed, optimality, cumulative violation and cost regret. - Evaluation of Context Incorporation: Simulation on integrating contextual information into the model. Comparison of methods (ignoring context, dividing data into buckets, composite kernel). Documentation of the impact of contextual information on performance. - Discussion and Conclusion: Insights on the most effective heating curve parametrization for different scenarios. Recommendations on the best algorithms for balancing safety and optimality. Future work directions for extending the model or improving user interactivity. **Requirement:** - We are seeking a student with experience in machine learning or optimization, who is enthusiastic about contributing to a project with practical applicability. - Prior knowledge in Gaussian Process (GP) and Bayesian Optimization (BO) is not expected but will be part of your literature studies. - Basic knowledge in python is mandatory and nice to have are conceptual ideas of digital twin and building energy simulation programs e.g. EnergyPlus. - Students need to find a supervisor at their home university.

Contact Details If this thesis looks interesting reach out to us and let’s chat about your background and explore how your skills align with our project: michael.locher@empa.ch

The project will investigate the potential of using Madupite to speed up the computation of Nash equilibria in DPGs. This requires developing a mature understanding of the theory of DPGs, as well as the solution techniques of Madupite, in order to identify how to best combine these tools. The main goal is to develop a publicly accessible code base that solves DPGs with Madupite acting as the inner MDP solver. This code base will be used to assess the speed-up with respect to existing benchmarks, as well as compute Nash equilibria in problem instances that were infeasible before. As a case study, we will compute equilibria in complex, multi-domain karma economies.

- Review the literature on DPGs and Madupite to have a mature understanding of these tools; - Develop a publicly accessible code base to solve DPGs with Madupite following best programming practices; - Test the developed solver on existing benchmarks and previously intractable instances.

Please apply directly to the posting on SiROP. We look forward to your application. Matilde Gargiani (gmatilde@ethz.ch), Ezzat Elokda (elokdae@ethz.ch)

This project aims to integrate the inhabitants’ behavior and its independence on real-time flexibility quantification and activation. If time allows, the student will have the opportunity to assess the methodology on measured data, from the NEST building at EMPA. 1. Familiarize yourself with the structure and content of previous works. Conduct a literature review on flexibility quantification of prosumers and inhabitants’ behavior modeling. 2. Propose a framework to model inhabitants’ behavior that can be integrated into the flexibility quantification formulation. 3. Assess the performances on a simplified case study and compare to results where inhabitants’ feedback is ignored. 4. Apply the methodology using data from the NEST building. 5. Summarize the main results and write a report.

Yi Guo (yi.guo@empa.ch) Julie Rousseau (jrousseau@ethz.ch)

This master thesis aims to develop a Model Predictive Controller (MPC) of a grid-interactive Thermally Activated Building System (TABS). The MPC will then be implemented in the HiLo living lab unit at the Nest building at Empa, Dübendorf. The master thesis will build up on an existing MPC and RC-model formulation of the HiLo TABS system.

The research question of this project is to evaluate the potential of a hydronic TABS for grid-reserve provision (primary, secondary, and tertiary reserves) for grid stabilisation & variable renewable energy (VRE) integration. Control variables are defined by the implemented TABS system in the HiLo Living Lab at the Nest building in Dübendorf. The thesis will consist of three work packages (WP): - **WP 1, MPC Formulation**: The student will first familiarize her/himself with the existing MPC of the HiLo unit developed at the chair. Subsequently, the MPC shall be modified, especially in the cost function formulation, to account for incentives of the TABS to respond to primary, secondary and tertiary reserves requests. For that, reserves request scenarios need to be implemented into the MPC. These scenarios will be defined in collaboration with SwissGrid and EWZ. - **WP 2, Simulation**: This WP will also build up on an existing RC model of the HiLo unit developed by the A/S group. After getting familiar with the existing model, the student will adapt it to represent a typical building type representative of the Swiss building stock. Special care in the model adaptation will need to be taken with respect to thermal storage capacity, heat propagation, and temperature phase shifting. Finally, the MPC from WP 1 will be executed on the RC model for both the typical building model and the HiLo building model, considering prior formulated electricity reserves scenarios. - **WP 3, Experiments (optional)**: In case of fast progress in the previous two work packages, the final optional work package will deal with the implementation of the grid-response MPC into the HiLo Living Lab at Nest, Dübendorf. As a first step, the student will need to get familiar with the HiLo controls interface, which will be introduced to the student by researchers of the group and of Nest. Subsequently, the student will calibrate the RC model with newly collected data based on short experiments. Finally, after successful implementation and calibration, a long-term experiment (around 2 full weeks) will be conducted, data collected, and the potential of the HiLo TABS to respond to reserves requests evaluated. **Candidate Description** We are looking for a highly motivated and self-organized candidate with a background in mechanical or electrical engineering, or architectural/civil engineering with strong interest in building controls. Proficient knowledge in at least one programming language (preferably Python or Matlab ) is essential for this project, as well as knowledge in mathematical optimization and building thermal dynamics. Prior experience in Model Predictive Control is desirable. Furthermore, you should enjoy numerical and experimental work. It is beneficial if you have prior experience in implementing building controllers. For this project, you will collaborate with researchers in our group. **References** - Silvestri, A., Coraci, D., Brandi, S., Capozzoli, A., Borkowski, E., Köhler, J., Wu, D., Zeilinger, M.N. and Schlueter, A., 2024. Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control. Applied Energy [Online], 368, p.123447. Available from: https://doi.org/10.1016/j.apenergy.2024.123447 - NEST HiLo website: https://www.empa.ch/web/nest/hilo

Alberto Silvestri (silvestri@arch.ethz.ch), Christoph Waibel (waibel@arch.ethz.ch)

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate

The project aims to categorize the extreme events and analysis their impact on flexibility provision particularly the demand side in the Swiss context. The envisioned steps for the project are as follows: 1) Literature Review: Conduct a thorough literature review to understand current methodologies and findings related to flexibility provision and the impact of extreme events on the power system. This review will help you identify gaps in the research and potential methodologies we can apply to our study. 2) Data Collection: Collect information on extreme events, including their type, duration, severity, and geo-graphical impact and variations. Any additional data on the fluctuations in flexibility provision (specific aspect) during past extreme events would be beneficial. 3) Categorization Framework: Categorize the scenarios based on several criteria, such as the probability of occurrence, potential severity, and expected socio-economic impact. 4) Quantitative Assessment: Use the collected data and chosen methodology to quantitatively assess the impact of extreme events on flexibility provision. It could be achieved through statistical analyses, machine learn-ing models, or simulation studies. 5) Case Studies: Conduct case studies on specific extreme events in Switzerland to provide a detailed analysis of the impact on flexibility provision. 6) Documentation: Document the entire process and findings in a comprehensive report.

Quantitively and Qualitatively define the following extreme events in the context of energy systems: (1) Heat Wave, (2) Cold Spell, (3) Financial Instability and, (4) Societal Change

Arnab Chatterjee - arnab.chatterjee@empa.ch

Water storage reservoirs are critical infrastructure for energy production (hydropower), water supply, and flood protection. The uncertainty of meteorological inputs is a major challenge to operating reservoirs to meet these (often conflicting) objectives. However, meteorological forecast skill has improved dramatically in recent decades, and forecast-informed reservoir operation (FIRO) is a promising approach to maximize reservoir benefits and avoid tradeoffs between conflicting objectives. The state-of-the-art for FIRO is a model predictive control (MPC) approach (see Figure 1, pdf attached): (1) forecasted meteorology is fed into a hydrological model that generates a forecasted river inflow to the reservoir; (2) reservoir operations are optimized for the forecasted inflow; (3) the optimized operations are applied for one stage (e.g., one day); and (4) a new meteorological forecast is received and the process is repeated. Yet, the hydrological processes that transform meteorology into river flow are complex and uncertain themselves. The two-step process of optimizing hydrological model parameters to historical observations and then optimizing reservoir operations to model-forecasted river flow may inadvertently yield suboptimal operating policies. This project proposes an alternative, data-driven approach—rather than attempting to model the complex dynamics between weather forecasts/measurements and reservoir river inflow, the data-driven approach will learn these dynamics from data (see Figure 2, pdf attached). That is, a Data-Driven Predictive Controller (DDPC) will use the learned dynamics to make reservoir release decisions. Specifically, the proposed DDPC controller will use a Transient-Predictor DDPC. The Transient Predictor, recently proposed by Institute for Automatic Control (IFA), improves upon prior DDPC methods by making causal multi-step predictions—inputs at timestep k do not affect outputs prior to time k. In addition to the Transient Predictor, proposed DDPC controller will also include an optimal regularization term that has been derived in previous work. This optimal regularization term allows the DDPC to account for system uncertainty due to the noise in the data, favoring control actions that align with the true system dynamics. DDPC reservoir operation is a novel concept and Transient Predictor DDPC is a new control method. Thus, this thesis seeks to make a notable contribution to data-driven reservoir management. A successful thesis may be converted into a publication in a high impact conference or journal. (see pdf for further details)

The primary objectives of the project are to: • Develop a reproducible workflow for applying DDPC to reservoir operation (considering the particularities of the hydrological context). • Demonstrate the application of DDPC for a testbed reservoir. • Identify opportunities and obstacles for further application of DDPC to reservoir operation.

Please send your resume/CV and transcript of records via email to Kevin Wallington (kwallington@ethz.ch) and Keith Moffat (kmoffat@ethz.ch). Competitive candidates will demonstrate multiple (but not necessarily all) of the following attributes: - Strong mathematical background - Experience in the field of control systems or related fields (inside or outside of the classroom) - Prior research or work experience that demonstrate the ability to learn and problem-solve independently - Programming skills - Written and verbal communication skills

Tasks: • Gain an understanding of current urban development strategies, their potential impacts on energy demand, emissions and well-being in an LMIC urban case study area. • Develop an Agent-Based Model to simulate urban development scenarios, focusing on novel urban development strategies (e.g., 15-minute cities) compared to conventional planning approaches. • Quantify and compare the emissions and energy demand impacts of the different urban design strategies. • Identify possible societal benefits of different urban development strategies, such as reduced commute times, proximity to essential services, and mitigation of mobility poverty.

Identify opportunities and formulate recommendations to reduce urban energy demand and emissions in the case study area while supporting societal well-being through novel urban planning concepts.

Dr. Mashael Yazdanie mashael.yazdanie@empa.ch

The aim of this project is twofold: i) design a safe reinforcement learning (SRL)-based controller for the optimal vehicle-to-everything (V2X) operation of EV fleets with PV-battery charging stations, and ii) analyze the performance of the SRL-based controller against existing physics-based controllers. The tasks that will be carried out in this project are: 1. Conduct a brief review on V2G/V2X operation of EV fleets for demand-side flexibility provision and existing control strategies, focusing on the SRL approach; 2. Design the SRL based-controller for EV charging scheduling for reduction of the station energy consumption and provision of flexibility services while satisfying the technical and operational constraints of EV batteries and charging stations; 3. Analyse the performance of the SRL based controller against physics-based models in terms of energy cost reduction, constraint violation and revenue generation; 4. Analyze the findings and write a report.

federica.bellizio@empa.ch mina.montazeri@empa.ch

The primary objective of this project is to develop safe deep reinforcement learning algorithms for building temperature control. While existing literature explores the use of DRLcontrollers for building management, these typically do not account for operational constraints. In contrast, our research focuses on designing a controller that adheres to specific constraints, such as maintaining a smooth policy and limiting the number of on-off switches within a given timeframe. Our approach involves employing various safe reinforcement learning techniques to design the controller and conducting a comparative analysis of their effectiveness.

developing safe deep reinforcement learning algorithms for building temperature control.

mina.montazeri@empa.ch

The primary objective of this project is to develop multi-agent reinforcement learning (MARL) algorithms for building control. While existing literature explores the use of Deep Reinforcement Learning(DRL) controllers for various building variables, these typically do not consider the interactions between different control variables and often design controllers independently for each variable or sub-system. Our research aims to address this gap by designing controllers that simultaneously manage all interdependent control variables. This approach shall involve utilizing multi-agent reinforcement learning to design these integrated controllers and conducting a comparative analysis with independently designed controllers using DRL. This will allow us to evaluate the effectiveness of the approach in optimizing building energy efficiency and performance.

developing multi-agent reinforcement learning algorithms for building control.

mina.montazeri@empa.ch

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate

This project is set to limited visibility by its publisher. To see the project description you need to log in at SiROP. Please follow these instructions: - Click link "Open this project..." below. - Log in to SiROP using your university login or create an account to see the details. If your affiliation is not created automatically, please follow these instructions: http://bit.ly/sirop-affiliate