As part of my AOSP tinkering, I wanted to give the Cuttlefish Virtual Device a try. While things seemed surprisingly well documented, the process of setting things up was not as smooth as I hoped. This is a short write-up of my experiences on my Ubuntu 24.04 system.
Build and installation
The build process is actually quite straight-forward. We pull the sources and build the lunch target aosp_cf_x86_64_only_phone-aosp_current-userdebug. Since I’m trying this on my laptop, I opted for android-16.0.0_r4. Android 17 builds are significantly more memory hungry and I max out my 32GB RAM quickly, making the build even slower than it already is. Android 16 on the other hand is content with less system memory.
repo init -u https://android.googlesource.com/platform/manifest -b android-16.0.0_r4 --depth=1
repo sync -c -j8
lunch aosp_cf_x86_64_only_phone-aosp_current-userdebug
m -j12
Interestingly enough, this make fails towards the end despite other AOSP builds finishing fine on the same machine. The issue seems to be related to nsjail. This GitHub issue describes a workaround we can use:
sudo sysctl -w kernel.apparmor_restrict_unprivileged_unconfined=0
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
Just be aware that this has security implications, since it disables AppArmor restrictions.
Next up is the installation of some host packages. The documentation provides instructions on how to build them ourselves, but we can also just use pre-builts, as described in the android-cuttlefish GitHub repo:
sudo curl -fsSL https://us-apt.pkg.dev/doc/repo-signing-key.gpg \
-o /etc/apt/trusted.gpg.d/artifact-registry.asc
sudo chmod a+r /etc/apt/trusted.gpg.d/artifact-registry.asc
echo "deb https://us-apt.pkg.dev/projects/android-cuttlefish-artifacts android-cuttlefish main" \
| sudo tee -a /etc/apt/sources.list.d/artifact-registry.list
sudo apt update
sudo apt install cuttlefish-base cuttlefish-user cuttlefish-orchestration
The guide also suggests to download the cvd-host_package.tar.gz from the Android CI. So we do that and extract it somewhere where we don’t lose it.
tar -xvf cvd-host_package.tar.gz
sudo mv cvd-host_package /opt/aosp/
sudo chown -R anon:anon /opt/cvd-host_package
We still need to add our user to some groups and we should be good to go.
sudo usermod -aG kvm,cvdnetwork,render $USER
Keep in mind that this usually needs a reboot or at least a logout to apply.
Troubleshooting
The first launch attempt is not very successful. There are a few suspicious lines in the console logs, but one sticks out:
failed to create a PCI root hub: failed to create proxy device: Failed to configure tube: failed to receive packet: Connection reset by peer (os error 104)
A quick web search later, we end up on the Google issue tracker. The posts suggest a sandboxing issue. So we try again with --enable-sandbox=false.
This fixes the issue. But we run into the next problem:
ERROR crosvm::sys::linux::panic_hook] failed to capture stderr during panic: Too many open files (os error 24)
This one is more obvious. A simple call to ulimit to increase the allowed number of open files and the error goes away:
ulimit -n 1000000
Success! We can finally connect to the device. And adb devices even detects it. But there’s one last problem: input doesn’t work. Neither is the screen registering mouse input, nor do the controls on the side panel do anything. Disconnecting and re-connecting in the web UI also fails. The console logs look fine but the device seems to just become unresponsive after boot.
Turns out the culprit is the version of launch_cvd we’re using. The guide explicitly mentions “Always download the host package from the same build as your images.”. A quick check shows: I got the wrong one. Oops…
But the download is actually not needed at all. We get our own cvd-host_package as part of the AOSP build. So instead of re-downloading a different archive, we can just run from our out folder directly:
out/host/linux-x86/cvd-host_package/bin/launch_cvd -report_anonymous_usage_stats=n --enable-sandbox=false
Once we do that, Cuttlefish is finally up and running.