A trained policy on a microcontroller

A legged robot is controlled in layers. At the bottom, each joint has a motor controller that holds a commanded angle hundreds of times a second, often on a small microcontroller of its own. Above it sits whatever decides the angles. On a hobby biped that is the same Arduino or ESP32 that drives the servos: it plays a gait table and corrects it from an IMU. On research quadrupeds and humanoids a learned walking policy decides the angles, about fifty times a second, and it runs on an onboard computer such as a Jetson.

The policy lands on the bigger computer mostly because it is trained in Python on a GPU, and the simplest place to run the result is a machine that runs the same framework. The network itself is often small. A walking policy for a small biped needs a few thousand weights, and a step is a few small matrix multiplications that a microcontroller with a floating point unit runs in well under a millisecond. This article goes through what a policy needs on a chip, then follows one from a trained model in a FloMotion scene onto an emulated ESP32 that drives the same robot.

Chip requirements to run a policy

The walking policies from prompt to policy are small networks. A policy runs fifty times a second. Each step it reads the joint angles and speeds, the direction of gravity and the body's turn rate. It returns a target angle for every joint. The math is a few matrix multiplications with a tanh between layers. It needs no operating system and no GPU.

The chip needs enough flash for the weights and speed to finish each step in time. A walking policy for a 19 joint humanoid has two hidden layers of 256 by default. That is about 83,700 weights, 327 KB as float32, and about 83,000 multiply adds per step, which an ESP32 handles with room to spare. A network with two hidden layers of 32 has 3,187 weights, 12.4 KB, and fits an Arduino Mega. Training picks the size, so a policy meant for a small board is trained small from the start.

The policy is also limited to what the board can sense. In simulation a policy often reads the robot's speed over the ground and the height of its pelvis. The simulator knows both exactly, but an IMU on the robot cannot measure them. A policy for a board is trained with the onboard observation instead: joint angles and speeds, gravity in the body frame from the accelerometer, the turn rate from the gyro, and the phase of the motion. For a 19 joint humanoid that is 46 values per step.

The firmware

On the chip the policy is part of an ordinary Arduino sketch. The sketch reads its sensors with the driver libraries it would use anyway, hands the readings to the FloMotion library, and writes the targets to its servos:

#include <FloMotion.h>
FloMotion::Policy policy;

void setup() {
  if (!policy.load()) while (true) Serial.println("no policy in flash");
}

void loop() {
  for (uint16_t i = 0; i < policy.jointCount(); i++) policy.setJoint(i, angleRad[i], velocityRadS[i]);
  policy.setImu(gravityX, gravityY, gravityZ, gyroX, gyroY, gyroZ);
  if (policy.step(millis())) {
    for (uint16_t i = 0; i < policy.jointCount(); i++) servos.write(i, policy.targetDeg(i));
  }
}

The library does no input or output of its own. step tracks the phase of the motion from the clock, builds the observation in the layout the model was trained with, runs the network and adds the result to the reference pose of the motion. Joints that the model marks as played from the motion take their angle straight from the clip. A helper turns raw MPU6050 counts into the gravity direction and turn rate the observation expects.

The core is plain C99 with no memory allocation, and the weights are read in place from flash. The Arduino wrapper around it knows where the model sits on each board family. The library source is public, in the flomotion-inference repository.

The model file

The trained model leaves the scene as one file in the safetensors format: a short JSON header with the name, shape and position of each tensor, followed by the raw numbers. The header metadata carries what the chip needs besides the weights: the observation layout, the control period, the joint order, the joints played from the clip and the clip's rate. The clip itself is stored as 16 bit joint angles, one row per frame.

The file is stored at a fixed place in flash that the library reads at boot. On an ESP32 it is a 1 MB flomotion data partition, which the build adds to the partition table of any ESP32 codebase that lists the library. On a Raspberry Pi Pico and on the AVR boards it is a fixed offset in program flash. The model is a file, not part of the program, so one build runs any model trained for the same joints. Swapping a model is a restart, not a rebuild.

Not every model fits every board. The model details show a fit table for each board: how much flash the weights and the clip take and how much the library has room for, the working memory, and an estimated step time next to the control period.

The firmware as the robot's controller

The same firmware can also drive the robot in a FloMotion scene, before any real board is flashed. The codebase is built on the server with the board vendor's Arduino core, and the resulting program runs on an emulated chip in the browser, the same emulators the circuit simulation uses. The chip does not know it is in a simulation.

A short mapping script sits between the chip and the robot. The agent writes it at the same time as the sketch. It watches what the chip does on its pins and buses and answers with what the robot's sensors would read. For a servo on a pin, it maps the pulse width, 544 to 2400 µs, onto the joint's range. For a PCA9685 servo driver at I2C address 0x40, it reads the channel registers the chip writes. For an MPU6050 at 0x68, it fills the registers from the body's gravity direction and turn rate, scaled the way the sketch's driver library expects. The mapping is the only part that knows what is wired where.

The scene writes the model file into the emulated chip's flash before boot. It is stored at the same place as on a real board, and the firmware loads it the same way. If the robot walks in the scene, the sketch, the library and the model file work together, and they can be flashed onto the real board unchanged.

The Firmware row in the Motions tab shows how fast the emulated chip runs against real time. It also shows how many servo pulses per second each pin produces and how many I2C transactions each address sees. A chip that should refresh its servos fifty times a second but manages thirty is missing control steps. A smaller network or a lower control rate fixes that.

Training for a board

The steps from a motion to a trained model are the ones in prompt to policy. For a board, two training settings change: the onboard observation, and for a small board a smaller hidden layer. The agent runs the whole chain from the chat: it trains the model, writes or reads the sketch, writes the mapping, attaches the firmware to the robot with the model, and reads the run's numbers. It handles the common sensor and servo setups well in practice, but a mapping that reads the wrong register or a sign flipped on an axis shows up as a robot that falls, so the run is worth watching before the board is flashed.

Each trained policy has a model details view. It opens with the Details link under the policy in the Motions tab. The codebase has a Flash group that lists the model files scenes write for its firmware, and each entry opens the same view. The view shows the model's facts, its layers, how far each joint may move away from the reference motion, the reference motion itself and the board fit, each with a short explanation.

Limits

• The emulated ESP32 and Pico run at about real time or slower, so a scene driven by firmware plays slower than one driven by a script or a policy in the browser. The step time on the fit table is an estimate until the chip runs.
• Real servos respond differently from the simulated actuators the policy trained against. A policy that walks in the scene is worth trying on the robot, but it is no proof that it walks there.
• Without joint feedback the sketch reports the last target as the joint angle and its change as the speed, which is common on hobby bipeds and less precise than measured angles.
• An Uno or a Nano has about 13.5 KB of flash free for the model file. That fits the weights of a small network, but not the weights and the reference motion of a full humanoid: a 32 by 32 model for a 19 joint H1 needs 17.2 KB.

Summary

A small control policy fits on the microcontroller that drives a robot's servos. The policy is trained with inputs the board can measure and a network sized for the board. An Arduino sketch runs it through the FloMotion library, which reads the model file from a fixed place in flash. In a FloMotion scene the same firmware drives the robot on an emulated chip with the model file in its flash. The sketch, the library and the model are tested together there before the board is flashed.

Links

• flomotion-inference, the FloMotion library with the C inference core, MIT licensed.
• safetensors, the file format of the model file.
• avr8js and rp2040js, the open source AVR and RP2040 emulators from Wokwi.