← Back

05 — Genetic Algorithm, Multi-Objective Reinforcement Learning · Research

Balancing Traffic Signals with Evolution

Presenting the thesis at the defense

01 / 02 — The trained agent in simulation, next to the Pareto front

Overview

A multi-objective traffic signal controller that pairs NSGA-II, a genetic algorithm, with an LSTM-DQN reinforcement learning agent. Instead of hand-picking how much throughput, fairness and smoothness should matter, NSGA-II evolves the reward weights the agent trains on, and returns a Pareto front of policies, each a different trade-off. The work was published in Procedia Computer Science as part of the 11th International Conference on Computer Science and Computational Intelligence 2026.

The problem

Fixed-time control (Webster, 1958) runs a pre-set timer and breaks the moment traffic stops matching history. Adaptive systems like SCOOT and SCATS react to live sensors but still lean on calibrated models. Deep RL adapts in real time, but it inherited the single objective. Real traffic is three goals pulling against each other: throughput (move the most cars), fairness (no lane waits forever) and smoothness (avoid stop-and-go waves). The usual fix is a hand-weighted sum like 0.7 · throughput + 0.2 · fairness + 0.1 · smoothness, which means guessing the weights and only ever exploring the one strategy they point to. An earlier study (Leal, 2023) used NSGA-II for traffic too, but only to optimise static signal timings.

Building it

The controller is a two-level loop. NSGA-II proposes a population of weight vectors [w1, w2, w3] (normalised to sum to 1); for each one an LSTM-DQN agent is trained on the reward R = w1·r1 + w2·r2 + w3·r3, run in the simulator, and scored on three fitness objectives; NSGA-II uses those scores to evolve the next generation.

The two-level training loop NSGA-II keeps a population of 30 reward weight vectors. For each, an LSTM-DQN agent is trained in SUMO, evaluated on throughput, queue variance and total stops, and the population is then selected and bred for the next of 20 generations. Inside each training run, the agent reads the halting cars on each approach, picks which approach goes green, and is rewarded by the weighted sum of throughput, fairness and smoothness. The output is a Pareto front of policies. Upper level · NSGA-II next generation, ×20 Weight vectors w = (w1, w2, w3)population of 30 Train an agent LSTM-DQN onreward R(w) Evaluate in SUMO throughput, queuevariance, total stops Select & breed non-dominated sort,SBX, mutation Lower level · for each w SUMO + TraCI BSD intersection LSTM-DQN remembers recentstates state halting cars N, S, E, W action which approach goes green R = w1·r1 + w2·r2 + w3·r3 r1 cars cleared · r2 fairness · r3 smoothness Output · Pareto front throughput ↑ queue variance → balanced
NSGA-II evolves the reward weights; for each set, an LSTM-DQN agent learns to run the intersection.

Everything runs in SUMO through TraCI, modelled on a real intersection: BSD Raya Utama × BSD Grand Boulevard in BSD City, with traffic from four approaches and demand that changes over time. The agent’s state is the count of halting vehicles on each approach, its four actions pick which approach gets the next green, and the LSTM gives it memory, since a single snapshot never shows the whole picture. The rewards map onto the three goals: r1 cars cleared, r2 the negative standard deviation of queue lengths, r3 the negative number of stops. The fitness objectives are throughput, queue variance and total stops.

NSGA-II ran with a population of 30 over 20 generations, binary tournament selection, SBX crossover at 90% and polynomial mutation at 1/n. To make the agent train at all, I seeded the replay buffer from a rule-based actuated controller and rescaled the reward terms, so jam penalties didn’t swamp every gradient. After a grid search: batch 256, learning rate 5e-4, discount 0.95, gradient clipping at 1.0. Each policy was evaluated over 1,000 steps with a 10–60 s green window and a 6 s yellow; the 60 s cap stops a greedy agent holding one light forever.

Outcome

I compared the NSGA-II balanced policy against four baselines, and it was best on all three metrics at once:

MethodThroughput ↑Queue variance ↓Total stops ↓
Fixed-time1,0116735,913
Rule-based actuated1,0152020,429
RL, max throughput1,0322120,977
RL, max fairness62539378,822
NSGA-II balanced1,0591314,676

Against the best baseline numbers, that is 35% lower queue variance and 28% fewer stops. The single-objective agents show why: max-throughput barely beat a simple actuated controller and stopped more often, while max-fairness switched phases so aggressively it caused gridlock.

The Pareto front shows how the weights trade off against each other:

Part of the frontThroughput weight w1ThroughputQueue varianceTotal stops
Throughput-heavy0.60 or morehighabove 12.5over 15,200
Balanced0.50–0.55about 1,050as low as 11.0about 14,400
Smoothness-leaningabout 0.38about 1,059, the highestthe highest–

The surprise is the last row: the highest throughput came from a policy that cared more about smooth flow than about throughput. Training took about 2 hours on a high-end laptop, and inference afterwards is cheap. SUMO and network training are stochastic, so exact numbers vary between runs, but the pattern over the baselines holds.

What I learned

The headline lesson is that chasing an objective directly is often the worst way to reach it: the policy that moved the most cars cared least about moving cars and most about smooth flow. Single-objective RL has a ceiling, and sometimes a cliff. Reward scaling turned out to be the difference between an agent that learned and one that didn’t. And the Pareto front was more useful than any single policy on it, since an operator can run a high-throughput policy at rush hour and a smoother one off-peak. It’s also the meeting point of my earlier work: where AV Simulation used a genetic algorithm to evolve the controller itself, here evolution shapes what a gradient-based RL agent is rewarded for. The next step is scaling from one intersection to a network of coordinating agents (multi-agent RL), with the coordination problem and compute bill that brings.

Next project CitiFlowAI