Neuro-Drones: Watch Neural Networks Learn to Fly and Fight

▶ Run it right here

What you are watching

Neuro-Drones is a demonstration of neuroevolution: training neural networks by selection and mutation instead of by gradients. It is meant to make learning visible. Every drone is flown by its own network, every decision it makes is drawn in real time, and the learning curve updates after each generation, so you can watch a population go from random behavior to competent flight without anything being scripted.

There are two modes. Race is a single population learning to fly a course of rings. Battle is two populations, red and blue, learning to fight each other, which turns learning into an arms race. Switch between them with the buttons in the top-left panel.

Race mode: learning to fly

Sixty quadcopters launch from a pad toward a course of 24 rings. Each drone flies a simplified quadcopter model, the way a hobby drone does in "angle mode": thrust along its tilted body, gravity, drag, and an altitude assist that makes zero throttle a hover.

• The brain is a small network: 18 inputs, two hidden layers of 14 and 10 neurons, and 4 outputs (throttle, tilt forward, tilt sideways, turn). That is 460 numbers in total, and those 460 numbers are the drone's entire skill.
• What it senses, all measured from its own point of view: the direction and distance of the next ring and the one after it, its own speed, altitude and tilt, and five range sensors that report how close the nearest obstacle is ahead, left, right, low ahead and straight down.
• How it is scored: 100 points per ring, partial credit for progress toward the next one, a bonus for speed, and a penalty for crashing.

When every drone has crashed, finished or run out of time, the generation ends and the next one is bred:

• The best 6 brains are copied unchanged, so a good solution is never lost.
• Parents are picked by small tournaments: 7 drones at random, the best one wins.
• A child inherits each neuron whole from one parent or the other, then about 3% of its numbers are nudged at random. Those nudges are where new skills come from.
• Two brand-new random brains join each generation to keep the population varied, and the size of the nudges adapts: smaller when the swarm keeps setting records, larger when it is stuck.

Every 6 generations the course is redrawn, so the swarm has to learn how to fly rather than memorize one route. Once any drone passes 12 rings, the swarm levels up: sharper turns and more pillars, then wind gusts from level 3 and pillars that slide across the course from level 4. This is a curriculum, the same idea as teaching easy material first.

Battle mode: learning to fight

Twenty red and twenty blue drones start on opposite sides of a round arena with mirrored cover pillars, so neither side has an advantage. They fly with the same physics as race mode and also carry a laser: the brain aims it up or down and decides when to fire, and to aim left or right a drone has to turn its whole body. Each drone can take 3 hits. Pillars block shots, and a force-field wall and ceiling push drones back into the arena. A round ends when one team is wiped out or after 45 seconds.

Each team evolves separately. Red only breeds from red, and blue only from blue. Each team's world is the other team, so when one side finds a better trick, the other side's scores drop until it finds an answer. This is called co-evolution, and it is why the score between the two teams can swing back and forth for a long time without either side getting worse.

A fighter is rewarded for hits (+10), kills (+20) and being on the winning team (+15), with small rewards for staying alive and for keeping its laser lined up on an enemy, and penalties for crashing, for pressing against the force field and, very slightly, for missing.

An autopilot that holds altitude. For a long time the drones had a habit you could see at a glance: they drifted up and stayed pressed against the ceiling, about two thirds of the time. The reason is simple. The ground destroys a drone, but the ceiling only holds it, perfectly level, above every pillar, for free. So drones that drifted up survived, and from the very first rounds evolution favoured climbing. Penalties for height, a ceiling that pushes drones down or destroys them, pillars as tall as the ceiling and a reward for staying near the ground did not fix it: the drones either stayed up or came down and stopped fighting. What worked is what real drones do. A real drone's flight controller holds altitude by itself, and the computer steering it only says which altitude to fly at. Battle drones now work the same way: the brain's throttle output picks an altitude between 6 and 64 metres, and a small controller flies there and holds it. Now no drone ends up pressed against the ceiling, fights happen at many heights, and one population even learned to fight low, at about 21 metres.

A hall of fame keeps the arms race honest. Co-evolution has a well-known failure: two teams can chase each other in circles, each forgetting how to beat strategies it already beat once. So every few rounds each team archives its champion, and every fourth round is a rematch: one team fights a squad of the other side's past champions, shown with gold health dots. Only the live team learns from a rematch, so it is pushed to keep beating its own history, and the share of rematches it wins is a progress measure that the score between the two teams cannot give.

The senses

A battle drone's inputs are built from a small library of senses. Each sense is a separate module that declares the inputs it provides and how to compute them, and the brain's input layer is sized from whichever senses are switched on. That makes it straightforward to add a new kind of sense later, or to compare two sets of senses against each other.

With every sense switched on, a battle brain has 42 inputs, hidden layers of 16 and 12 neurons, and 6 outputs: 970 numbers. The first five senses are the original set; tactics, hearing and light were added in version 0.8. When senses are added, brains that were already trained carry over: the new inputs start with zero weight, so the drones behave exactly as before until evolution finds a use for them.

The brain view also shows what each drone is paying attention to: next to every sense, a percentage and a bar for how much that sense is driving the brain at that moment, worked out from the strength of its connections times what is coming in through them.

With the sensor display switched on, the drone being followed shows what it perceives: gold lines for what its eyes see and purple lines for what its ears hear, each growing with the strength of the signal.

Toward real drones

The senses are modular for a reason: real drones carry real sensors, and the long-term direction is brains that could one day help fly real drones. So every sense now has a spec sheet for the hardware it stands for, with that hardware's update rate, delay, noise and chance of losing a reading:

Battles can run in three sensor conditions, chosen in the demo's senses menu: perfect (every reading exact and instant), realistic (each sensor reads only at its own rate, late, with noise, and sometimes loses a reading) and harsh (all of that worse, plus fog that dims what cameras see).

Does training with the flaws help? This is the central question for moving from a simulation to the real world. Teams trained with flawed sensors fought teams trained with perfect ones, with both fighting under the flaws. With realistic flaws, the teams trained on perfect sensors actually did better (62% of rounds). In harsh conditions a first batch of runs favoured training with the flaws (70%), but a second batch on fresh runs did not (46%), so the answer there is inconclusive. A likely reason is that these brains have no memory: they cannot learn to compensate for delays and lost readings they have no way to notice.

Senses that model real hardware. A further set of senses measures only what a real drone's sensors physically could. Pick them with the Hardware preset in the demo's senses menu:

• Ultrasonic: six short-range sonar cones (ahead, behind, left, right, up, down) out to 7 m, echoing off anything solid, including drones of either team.
• Lidar: a ring of twelve laser beams, precise out to 40 m. Fog shortens its reach.
• Radar: the two nearest drones ahead, with how fast they are closing in. It works in fog but cannot tell friend from foe.
• Team data link: the realistic replacement for x-ray sight. A drone only learns about an enemy that some teammate actually has on camera, five times a second, a quarter of a second late, with one message in ten lost.
• The drone's own motion estimate now drifts, the way real motion sensors and barometers slowly do.

With realistic sensor flaws on both sides, drones using only these hardware senses won 264 of 560 head-to-head rounds (47%) against drones with x-ray sight over 14 runs, once that x-ray sight was as late and noisy as a real data link would be. They lean most on their own motion and on lidar, and the radar, unable to tell friend from foe, ends up used less.

Sensors weigh something. Every sensor now has a realistic mass, modelled on small real hardware, and a heavier drone has less thrust to spare: the full set of hardware sensors adds 158 g to a 500 g drone and cuts its thrust-to-weight ratio from 2.2 to about 1.7, so it climbs and accelerates more slowly. That turns sensor choice into a real design trade-off. Measured against the fixed panel of reference opponents, leaving out the lidar and the radar saved 65 g and cost nothing measurable, while the lightest set, without the short-range ultrasonic sensors and microphones, scored about nine points lower (a hint, not yet significant).

When a sensor fails. Real sensors fail in flight, so any sensor can now be switched off in the middle of a battle, after which it reports "nothing detected". To find out which sensors a drone cannot do without, the same trained teams fought the fixed panel of reference opponents once with every sensor working and once with each sensor failed in turn. Because these are the same brains facing the same opponents, the comparison is far sharper than comparing separately trained teams. Losing the microphones hurt most: drones with hardware sensors lost about 20 points of their score and drones with every sense about 16, because sound carries around cover and comes from every direction. Next came the radar (about 13 points), even though it had looked little used when we measured where the drones' attention went. Losing the data link, the drone's own motion and status, or the lidar cost about 7 points each, while losing the ultrasonic sensors or the camera made no measurable difference. One lesson: brains depend on whatever they were trained with. Teams trained without lidar and radar did just as well, yet teams trained with them suffer when they fail. The natural next step, as in real robotics, is to train with random sensor failures.

Training with failures. Robotics teams make controllers robust by switching sensors off at random while training. We tried the same: in some training rounds, one random sensor of each team fails for the whole round. When it happened in half of all rounds, the drones did become less dependent on any single sensor (a failure cost them about 5 points instead of 9), but they also became weaker fighters overall, so even with a sensor down they scored no better. In one round in five, the first two batches of runs looked like the clearest improvement yet; a third batch reversed it, and over all 30 runs there is no reliable gain. So it stays an option to experiment with, switched off by default.

Wind. Real drones fight in wind, so realistic conditions now bring a gusty breeze of 2.5 metres per second, and harsh conditions 4.5. Each arena layout has its own wind direction, so neither side is always upwind. At these strengths, wind made no clear difference: drones trained in calm air lost only a few points in a gale, and training in wind did not help. The altitude-holding autopilot and the drones' own air resistance absorb most of it, and their opponents fly in the same wind.

A short memory did not help. The obvious fix for brains that cannot cope with delays is memory, so the brains got four memory cells: extra outputs whose values come back as inputs on the next step. In four tests, from scratch and added to already-trained teams, in realistic and harsh conditions, teams with memory never won more, and in harsh conditions they lost about 60% of rounds. The likely reason is that random mutations to those feedback connections echo through the loop and destabilise the brain. So the next idea was closer to how real autonomy software copes with delays: rather than hoping the brain remembers, tell it how old each reading is. The first batch of runs looked like the best result any new sense had produced (67% of rounds), but a second batch on fresh runs came out even (48%), so there is no reliable gain yet.

Measuring against a fixed yardstick. To answer questions reliably, results are now measured the way game-playing AI research does it: a fixed panel of sixteen reference teams, rebuilt identically every time, that every candidate has to fight from both sides, with a margin of error on every result. Measured that way, the tuning described above is confirmed (about 22 points better than the settings before it, with a margin of error that stays clear of zero), while the hall of fame's advantage and the timestamps' advantage are not.

The most useful lesson so far is about measurement itself. Four times now a striking early result has disappeared when repeated. When both teams learn at once, a batch of seven runs can swing by 15 to 20 percentage points by pure chance, which is why every result on this page is only reported once a second, independent batch has been run. The next step is making these measurements sharper, with more runs, longer training and a fixed panel of opponents to test against.

From the browser toward hardware. Any trained brain can now be exported as a single file of C code with its weights and one function that makes a decision: no memory allocation and nothing beyond the standard maths library, small enough for a microcontroller or a drone's companion computer. Every time the code changes, an automated build compiles that C code and checks it against the in-browser brain on 64 test cases; the two agree exactly. The file documents what each input means, which real sensor would provide it, and how the outputs map to a standard flight controller. The racing brain is the natural candidate for real hardware: its inputs are the direction to the next waypoint, the drone's own motion and five distance sensors, which makes it, in effect, an obstacle-avoiding waypoint follower. It would run behind a normal flight controller with a geofence and a kill switch, and be tested in a high-fidelity simulator first.

What we measured

All of these come from headless runs of the same simulation, with the commands recorded in the project so they can be repeated.

• Race, learning speed. Averaged over 10 independent runs of 48 generations, the best drone passed 11.6 rings per course over the last 12 generations, up from 6.6 before the curriculum and selection were tuned. Repeat runs differ by roughly one ring.
• Race, long runs. In 150-generation runs, 6 of 6 kept leveling up, reaching levels 11 to 20 with best flights of 9 to 18 rings.
• Battle, does it learn? The score between two co-evolving teams cannot show progress on its own, because both sides improve together. So trained teams were frozen and pitted against brand-new random teams, a different opponent and arena every round. After 60 rounds of training, trained teams won 296 of 300 of these rounds across 5 runs; random against random comes out about even.
• Battle, does the hall of fame help? Not proven. Head to head, teams trained with it first won 321 of 560 rounds (57%) over 14 runs, but against a fixed panel of reference opponents (below) teams with and without it scored the same (a difference of 0.3 points, well inside the margin of error). It stays on because it does no harm and makes the rematches visible.
• Battle, tuning by experiment. Settings change only when a head-to-head test says so. When the battle brain grew from 650 to 970 numbers, mutating 2% of them per child instead of 4% won 344 of 560 rounds (61%) over 14 runs, so 2% became the default. Stronger selection followed: picking parents from tournaments of 5 instead of 4 won 303 of 560 rounds over 14 runs and beat tournaments of 6 directly. The same kind of test kept the hall-of-fame rematch every 4 rounds (every 3 or every 6 both lost to it) and 3 champions copied unchanged per round (2 lost). Honestly, these gains added up to less than the single tests suggested: all of them together beat the earlier settings in 53% of rounds, because the variant that wins a noisy test tends to look better than it is. So settings now change only if a fresh batch of runs confirms the result. The first to pass was making each random change half as large, which won 63% of rounds over 14 runs, including 64% in its confirmation batch. Measured end to end, the battle settings at the end of that night beat the ones from its start in 416 of 560 rounds (74%), ahead in 12 of 14 runs.
• Battle, do the new senses help? Not yet, and the reason is now clear. Teams with all eight senses were trained alongside teams with only the original five, then fought each other from both sides in fresh arenas: over 10 runs the teams with the new senses won 163 of 400 rounds (41%), and with bigger teams of 40 a side 55 of 120 (46%). Giving already-trained teams the new senses and comparing them with the same teams carrying on without came out exactly even, 200 of 400: free, but no gain yet. The reason is that the original nearest-enemy sense is x-ray vision: it gives the exact direction of the nearest enemy straight through cover, so hearing and light mostly repeat what a drone already knows. Take x-ray vision away and drones still learn to fight with only hearing, light and line of sight (they beat brand-new random teams 394 times in 400), but they lose to x-ray-sighted teams about two rounds in three. Re-checked after the battle settings were tuned to learn much better: still no gain (243 of 560 rounds, 43%), which supports the x-ray explanation rather than weak learning.
• Battle, training with sensor flaws. Trained with realistic flaws vs trained with perfect sensors, both fighting with the flaws: 211 of 560 rounds (38%) for the flaw-trained teams over 14 runs. In harsh conditions, 324 of 560 (58%) but ahead in only 7 of 14 runs, after a first batch of 70% failed to repeat: inconclusive.
• Battle, do drones learn to use hearing and light at all? Yes, a little and consistently. Comparing what trained and untrained drones pay attention to, over 10 runs of 120 rounds, the share of the brain driven by hearing rose by 1.5 percentage points (in 8 of the 10 runs) and by light by 1.0 (also 8 of 10), while x-ray sight of the nearest enemy fell. The biggest gain went to the drone's own motion, up 4.4 points in every run: flying well matters most.

How to use it

• Click or tap any drone to follow it: the camera, the brain view and the family tree switch to that drone. Esc goes back to the leader.
• Speed: 1x, 3x, 10x and 40x. At 40x a generation takes seconds.
• Cameras: chase, cinematic, overview, top-down, side and free. C cycles them, V jumps to top-down, X to the side view, the mouse wheel or a two-finger pinch zooms, and dragging switches to the free camera.
• Smooth on any device: the graphics adjust themselves, lowering resolution and glow effects if your device struggles and raising them again when it can keep up.
• On a phone the panels fold away so the drones have the screen. The ⓘ button shows what is going on, and the three buttons in the control bar open the brain, the learning chart and the debug panel.
• Keyboard: Space pause, 1 to 4 speed, T trails, S senses, H shadows, B and F brain or family tab, D debug panel, N new course or end the round, M sound.
• Comfortable to watch: bright hit flashes are limited to about three per second, and if your device is set to reduce motion the demo switches to a calm mode with no bright flashes at all. Buttons have spoken labels for screen readers, and everything works from the keyboard.
• Experiment with the senses: in battle mode the senses menu switches each sense on or off. Try Realistic, which takes away x-ray sight of the nearest enemy, and watch your trained teams adapt. Experiments start from your saved teams and never overwrite them.
• Progress is saved in your browser after every generation, so closing the tab does not lose what the swarm has learned. Reset starts over. On a computer the save menu also downloads the swarm (race) or both teams (battle) as a file you can load again later or share.

How it is built

The neural network, the evolution and the flight physics are written from scratch in plain JavaScript, with no machine-learning library: every brain is one flat array of numbers, which is what lets evolution treat it as something to copy and mutate. The 3D view uses three.js. The simulation runs at a fixed 60 steps per second regardless of the frame rate, and the faster speeds run more steps per frame. Nothing is sent to a server; the only thing stored is the autosave in your own browser.

The project is tested the way production software is: unit tests for the network, the evolution, the physics and every sense, browser tests that drive the real page (including the phone layout), and fixed-seed fingerprints that prove a change to one mode leaves the other mode's simulation bit for bit unchanged.

Limits

• These brains are tiny and have no memory: each step they decide only from what they sense at that moment. They cannot plan, and in battle they cannot coordinate beyond what the tactics sense tells them.
• Neuroevolution is a good way to make learning visible, but it is not how large modern AI models are trained; those use gradient descent on far bigger networks. The principle on display, improvement by measured selection rather than by hand-written rules, is the part that carries over.
• Battle results are noisy. Each run of co-evolution settles on its own strategies, so two runs with identical settings can produce teams of very different strength. That is why the measurements above use many runs and fresh opponents.

Get new results as we publish them

Roughly monthly, one finding per email, in plain English first. Including the approaches that turned out not to work, which are usually the useful ones. No sales email.

Leave this empty Email address Subscribe

Double opt-in: we send a confirmation link and add nobody who does not click it. One-click unsubscribe on every email.