Wiring a Real Fly Brain Into a Drone

How this actually started

I did not wake up one day and decide to simulate a fly brain. Google and Janelia dropped MaleCNS v1.0, a full synapse-level map of the male fruit fly's brain, about 166,700 neurons and roughly 125 million synapses, all public under CC-BY. That announcement alone would have been a nice blog post to read and forget. What actually pulled me in was what happened on X in the week after.

Within days a handful of hobby projects showed up using the exact same dataset. Someone built a browser game where the connectome flies a fly through obstacles. Someone else ported it into a Minecraft mod, of all things, keeping the full 166,700 neurons and 25 million directed edges and having them drive a scripted body. Older extracts of the same data started getting dug up and reposted too.

None of this was corporate. It was just people grabbing a real brain off a public bucket and hooking it up to whatever body they had lying around, and posting clips of it. The timeline nerd-sniped itself, and I was one of the people who got sniped.

So I built fly-playground first: a Rust to WASM leaky integrate-and-fire simulation of the full connectome, running in the browser, driving a 3D fly through a small world with a live brain inspector next to it. Then, because a fly flying itself around a browser is fun but a real machine is more fun, I built fly-drone: the same frozen brain, camera pixels for eyes, and a learned decoder that turns its motor neuron activity into commands for a MuJoCo-simulated quadrotor. Genuinely one of the more satisfying builds I have done in a while, and also one of the most humbling, for reasons I will get into.

What the connectome gives you for free

The thing that makes this different from writing your own neural net is that you do not get to choose the wiring. MaleCNS ships every neuron with a body id, a measured 3D position (for 139,662 of the 166,700 cells, the rest stay in the simulation without a plotted position, nothing is faked), an anatomical group, and a predicted dominant neurotransmitter.

The pipeline keeps every neuron with a real superclass and every edge with at least 3 synaptic contacts, which leaves 166,700 neurons and 10,520,431 directed edges. That edge count is not a design choice I made, it is what the actual fly has, filtered only to drop noise-level contacts.

Sign comes from biology too, not from me: acetylcholine reads as excitatory, GABA and glutamate as inhibitory (the standard Drosophila GluClα assumption), and neuromodulatory sources like dopamine or serotonin keep their anatomical wiring but inject zero direct current in this model, since there is no principled way yet to turn "this synapse is dopaminergic" into a current value. That is a real modeling choice, documented as one, not hidden.

The point is: I did not design a brain that can see a light and turn toward it. That circuit, insofar as it exists, is already sitting in the data. My job was to get information in and out of it without lying to it about what it is.

The actual math, LIF style

Both projects run the same core: a discrete leaky integrate-and-fire model, one neuron state update per simulated tick. The continuous form is the textbook one from Gerstner and Kistler's Neuronal Dynamics:

Every symbol there is one of a handful of things:

• $v$ is the neuron's membrane potential, the one number that actually evolves over time.
• $\tau_m$, the membrane time constant (20 ms here), sets how fast $v$ forgets old input and decays back toward rest if nothing new arrives. Smaller $\tau_m$ means a leakier, more forgetful neuron.
• $v_{rest}$ is the potential the neuron relaxes to with zero input, taken as 0 in this model.
• $R$ is the membrane resistance, folded to 1 here so current and voltage share units and nothing needs converting.
• $I(t)$ is whatever current is arriving right now, from synapses, from injected sensory drive, or from noise.

with a spike and reset to $v_{reset}$ whenever $v \geq v_{th}$, the firing threshold. Taking $v_{rest}=0$, $R=1$, and integrating exactly over one 5 ms tick with the input held constant folds the leak into a single multiply:

$\Delta t$ is the simulation's tick length, 5 ms, fixed for the whole system, so $\lambda$ is just a single number computed once and reused every tick: whatever voltage a neuron is holding, 77.88% of it survives to the next tick before any new input is added. Then, for neuron $i$ on tick $t$:

$u_i(t)$ is the total new input that neuron picks up this tick, and it is just a sum of four independent sources: $I_{syn,i}$ is whatever arrived over synapses from neurons that fired last tick, $I_{inj,i}$ is externally injected current (this is the door sensory data walks through), $b_i$ is a small fixed per-neuron bias, and $\sigma \xi_i(t)$ is Gaussian noise of standard deviation $\sigma$, independently drawn per neuron per tick. Put the leak and the new input together and you get the entire state update:

decay the old voltage by $\lambda$, add this tick's input, and if that crosses 1.0, the neuron fires and resets to 0. That is the entire model, and it is small enough that seeing it as actual code is more convincing than seeing it as algebra. This is the real, unedited single-neuron step from brain-core, in Rust:

/// Pure single-neuron update. `input` already includes synaptic drive +
/// injected stimulus + noise for this tick. Order-independent across neurons
/// because the caller double-buffers `input`.
///
/// Returns `(new_v, fired, new_refrac)`. While refractory, `v` is clamped to
/// `v_reset`, no spike is emitted, and the refractory counter decrements.
pub fn integrate_one(v: f32, refrac: u16, input: f32, p: &LifParams) -> (f32, bool, u16) {
    if refrac > 0 {
        return (p.v_reset, false, refrac - 1);
    }
    let v_new = p.leak * v + input;
    if v_new >= p.v_threshold {
        (p.v_reset, true, p.refrac_ticks)
    } else {
        (v_new, false, 0)
    }
}

p.leak is exactly the $\lambda$ computed above, precomputed once from the millisecond time constants so the hot loop never calls exp per neuron per tick. Every one of the 166,700 neurons runs through this same nine-line function, once per tick, with nothing neuron-specific except its own v, its own refractory counter, and whatever input its synapses and sensory role delivered that tick. There is no separate code path for a visual neuron versus a motor neuron. The only thing that makes a cell "visual" is that something outside this function chooses to write current into its input slot.

The caller wraps this in a loop over the whole graph, and that loop is where the one-tick synaptic delay actually lives: a spike computed on this tick adds to a separate input_next buffer, which only becomes input_cur after every neuron has finished this tick's update. Every synapse therefore carries exactly one tick of delay, so a spike lands in the next tick's input buffer, never the current one. That single design decision is what makes the whole thing deterministic and order-independent within a tick, and it means a signal traveling h hops through the graph physically cannot arrive faster than h ticks. It also means the frozen brain has a real notion of how far away things "should" propagate, which becomes important later.

I like worked examples more than I like being told a formula is correct, so here is one straight from the docs. There is a small tonic bias, 0.85, applied to the wing-power motor neurons to keep the fly's wings beating even with no sensory input. Starting from reset:

So the cell fires exactly every second tick, a rate of 100 Hz, mean spike value 0.5.

Nothing downstream reads raw spikes directly, since a 0-or-1 signal every 5 ms is too jittery to steer a motor with. Instead every neuron feeds an exponential moving average of its own spikes, the activity trace, updated every tick as

where $s_i(t)$ is the neuron's spike, 0 or 1, on that tick. A cell spiking every other tick has a long-run mean spike value of 0.5, and an EMA tracks the mean of what it is fed, so the trace converges to 0.5 as well, and this is the number everything else in the system, readouts, the policy's own inputs, actually consumes. And when you go measure it in the actual running sim with no light and no motion, the recorded thrust readout in the dark is 0.5031. That 0.003 gap is just the injected noise and whatever synaptic drift leaks in. Watching a value predicted purely from algebra show up in a live simulation to three decimal places is, honestly, the moment this stopped feeling like a toy to me.

Getting a camera into a brain that never had one

Here is the actual hard problem, and it is not the neuron math. The connectome has real visual neurons, Mi1 and Tm3 cells for light, LC4 and LPLC2 for looming, all anatomically identified. What it does not have is a formal specification for what a pixel means to those cells, because nobody has fully solved fly vision at that resolution yet.

So the sensory front end in fly-drone is explicitly an engineered adapter, not a claim about biology. Two cameras render 64 by 48 images, and per eye we compute Rec. 709 luma per pixel and reduce it to two scalars: a sustained bright-excess statistic and a frame-over-frame dark-area-growth statistic, one crude proxy for looming.

Those four currents (left/right light, left/right loom) get injected directly into the annotated cell groups every 40 ms, and everything downstream of that injection point is the untouched, frozen connectome. Nothing about the encoder is neuroscience. It is calibrated to this exact camera setup and this exact scene, and changing the camera geometry literally changes an "encoder version" hash that invalidates every trained decoder, because the meaning of a cue changes with it.

A held current of size $c$ is exactly the $u_i(t)$ from the LIF equation above with no other input, so plugging it into the same fixed-point logic from the tonic-bias example says the cell reaches a steady value of $c/(1-\lambda) \approx 4.52c$, and starts firing on every single tick once $c \geq 1$. So a "cue of 2" is not an arbitrary UI number, it maps directly onto how hard that specific population of cells is being driven, through the exact same math as the wing motor neurons.

The best debugging story from this whole project happened right here. The two cameras splay outward so their fields overlap slightly in the middle, letting the sign of left-minus-right brightness tell you which side a target is on.

My first splay angle, 0.45 radians, seemed reasonable. It was wrong in a way that took actual measurement to catch: with that much overlap, a target anywhere within about 0.3 radians of center sat inside both eyes' views at once, both cues saturated, and the tiny residual difference between them had the wrong sign, because the farther eye's view of the target's edge reads brighter than the nearer eye's more centered, darker read of it.

Any decoder trained on that setup learned to stop turning right around that dead zone, every single time, because the training signal itself was lying near center. Widening the splay to 0.75 radians removed the dead zone entirely and flipped every sign error in the measurement table. That is a full afternoon lost to what looked, from the outside, like a policy that just would not learn to finish its turn. It was never the policy. It was the eyes.

Reading the brain back out, and training something to listen to it

The output side has the same shape of problem, in reverse. The policy that flies the drone is only allowed to see the activity of 2,022 descending and motor neurons, the actual anatomical output pathway of the fly's brain, expressed as spike-rate traces between 0 and 1. No pixels, no pose, no target position, nothing simulator-privileged. Just: what is the brain's own output doing right now.

On top of that we train a tiny decoder, two hidden layers of 32 units, that maps those 2,022 numbers to four numbers: forward, lateral, vertical velocity, and yaw rate. Training happens in two stages. First a supervised warm start: hold a visual target at a known bearing, record the resulting neural features, and fit the decoder to a simple proportional yaw law with regression. That gives PPO (Schulman et al.'s 2017 clipped-objective policy gradient method, with GAE for the advantage estimate) a sane starting point instead of making it discover steering from scratch inside 2,022 noisy dimensions.

The reward itself needed a shaping term to avoid a lazy optimum. With $\beta_t$ the target bearing error at tick $t$, a plain

pays a policy for already facing the target without ever teaching it to turn toward one that starts off to the side, since a policy sitting at a fixed, mediocre $\beta$ collects the same reward every tick whether it is closing the gap or not. The fix is a potential-based shaping term on top of it, keyed to how much the bearing error actually shrank since the last tick:

Summed over a whole episode this telescopes to $10\,(|\beta_0| - |\beta_T|)$, exactly ten times the total bearing reduction from start to finish, and a policy cannot farm it by oscillating back and forth, since turning away costs precisely what turning back later earns. Ng, Harada, and Russell showed in 1999 that adding a term of this exact shape never changes what the optimal policy actually is, it only changes how fast an imperfect learner finds it, which is a nice property to be able to prove rather than just hope for.

None of that training touches the brain. PPO updates the decoder's weights only. The connectome, its 10.5 million edges, its signs, its tonic biases, stay exactly as measured, for every single training step.

That constraint is the entire point of the project for me: I am not building an agent from scratch and calling it "fly-like." I am trying to find out how much a real, measured nervous system can be made to do when you are honest about only touching the parts you are allowed to touch.

Before any of the RL training is even allowed to start, there is a causal gate: synthetic left-versus-right stimulation has to actually produce different policy features, silencing the visual cells has to collapse that difference, and the whole thing has to hold up again with real rendered camera frames, not just synthetic test inputs.

If that gate fails, training refuses to run. It exists because it is genuinely easy to build a rig where the "brain" is technically wired in but the decoder has quietly learned to ignore it in favor of some other leak in the pipeline, and I wanted a hard check that could not be argued with.

Where the training actually stands right now

I would rather show the real numbers than describe this as further along than it is. The first held-out evaluation, run with the old 0.45 radian splay, 50 seeds, warm start plus 1,024 PPO steps, landed the trained decoder at 40% success turning toward the target, against 0% with zeroed features, 0% with the visual cells silenced, and 12% with the features shuffled.

Fifty trials is not a lot to hang a percentage on, so it is worth actually showing the uncertainty rather than quoting a bare 40%. For $\hat p = k/n$ successes, the Wilson 95% interval is

which at $n=50$, $\hat p = 0.40$ (20 successes) works out to a fairly wide [0.28, 0.54]. So I am not reading too much into the exact number, but the shape of the ablation gap is the part that matters more than the point estimate: the trained decoder clearly beats every condition where it is denied real, correctly-identified sensory input, which means the 40% it does get is coming from the connectome and not from some accidental leak elsewhere in the pipeline.

Every one of the 30 failures stopped turning at almost exactly the same bearing, around 0.25 to 0.36 radians off target, which is the tell that sent me back into the sensory encoder and found the splay dead zone described above rather than a training hyperparameter. Retraining on the corrected 0.75 radian splay is in progress as I write this, and I expect that number to move once it finishes, since the dead zone that capped it is now gone from the input side.

The performance side is in better shape. A full neural tick over all 166,700 neurons runs at 2.10 ms p50 under normal load, rising to about 3.9 ms p50 and 5.1 ms p95 once the rest of the frame is competing for the CPU, against a 5 ms real-time budget per tick.

One full camera frame, 8 neural ticks plus 40 physics substeps plus rendering both eyes, currently costs 32.3 ms end to end (22.6 ms of that is the brain, 6.2 ms is the two eye renders, the rest is physics), which works out to running about 1.24 times real time on a single desktop thread. That number used to be 0.72 times real time, below real-time, until I found that the default MuJoCo render settings, a 640 by 480 buffer with 4x multisampling and floor reflections on, were costing 28 ms per frame for two 64 by 48 images nobody needed at that fidelity.

Dropping multisampling and reflections and rendering at the actual target resolution cut eye rendering to 6.2 ms and was the single biggest performance fix in the whole project, bigger than anything I did to the neural code itself. The brain now genuinely dominates frame cost, close to 70% of it, almost entirely spent drawing 166,700 Gaussian noise samples every single tick, which is a satisfying thing to know for certain rather than guess at.

The looming and obstacle-avoidance task has not been trained yet at all, it is still queued behind getting the visual steering number up on the corrected encoder. I'd rather say that plainly than let the architecture talk carry more confidence than the results currently support.

Where it actually stands

This is a research sim, not flight software, and I want to be upfront about that rather than let the framing oversell it. The stabilizer that keeps the drone level uses ideal simulated state, not a real IMU. The visual adapter is two calibrated numbers per eye, nothing close to actual fly vision. Running the full connectome onboard a real airframe is its own hardware problem I have not touched yet.

What I do have is a real anatomical brain, frozen and unmodified, whose measured output can be shown, with ablation tests, to be doing real work: performance drops when you zero the features, drops when you silence the actual sensory cells, drops when you shuffle which feature is which. It is not decorative.

The wider thing I took away from this week is less technical. A public dataset dropped, and within days a small, mostly uncoordinated crowd on X had turned it into games, mods, and sims, purely because it was fun and freely available. That is a genuinely good version of what a viral moment on that platform can produce, and I am glad I let myself get pulled into it instead of just liking a few posts and moving on.