Running Switch games natively on iOS/macOS

14 min read

Last week I posted Mario Kart 8 Deluxe running on my iPhone, and the question I got the most was “what emulator is this ?”

well… none ! the game's own code runs directly on the iPhone's CPU. So here's how it actually works, in (a lot of) detail, and how it went from a Mario Kart port (Mush8) to a small toolkit (nxhost) that now runs Sonic too (Velocity)

Wait, no emulator ?

The Switch runs on an NVIDIA Tegra X1, which is an arm64 chip. Apple Silicon (your Mac, your iPhone) is arm64 too. So the game's machine code is already something the iPhone's CPU understands, there's nothing to translate : you can (almost) just load it in memory and jump into it

Almost, because the game thinks it's running on a Switch, and a few instructions only make sense there :

  • x18 : on the Switch it's a normal register, games use it for whatever they want. On Apple platforms x18 is reserved by the system and can get wiped at any moment, so a game using it would randomly break
  • svc : that's how a game talks to the Switch's kernel (create a thread, map some memory, wait on something…). Run that on an iPhone and you're asking Darwin to do Horizon's job, which, yeah, doesn't go well
  • system registers : the thread pointer, the timer, the cache info… the Switch's kernel sets them up for the game, but iOS either uses them for its own stuff or gives different values

Everything else (and that's like 99.9% of the game) runs as is

nxpatch : fixing those instructions ahead of time

nxpatch goes through the game's modules (the NSOs : main, subsdk0, subsdk1…) once, before the game ever runs, and rewrites every one of those instructions. It's all in rewrite.rs

Trampolines

The problem with patching arm64 code is that every instruction is exactly 4 bytes. You can't just replace one instruction with three, everything after it would move and every branch in the game would point to the wrong place

So each instruction that needs fixing gets overwritten with a single b (a branch) to a trampoline : a small piece of code nxpatch writes at the end of the module, which does the work and then branches back right after the original instruction. The game doesn't move at all, it just takes a little detour

Right before the trampolines, nxpatch also reserves a page I call the keypage. nxhost fills it when it loads the game, and trampolines read from it : where to find the current thread's context, the address to call for system calls, the timer multiplier… That way nxpatch doesn't need to know any nxhost address when it patches

x18

x18 is the big one, games use it everywhere. Since we can't trust the real x18 to keep its value, every thread gets a “virtual” x18 that lives in its context in memory, and each instruction touching x18 becomes a trampoline that roughly looks like this (here for ldr x0, [x18, #0x10]) :

stp  x9, x10, [sp, #-16]      // save two scratch registers
mrs  x9, tpidrro_el0          // Apple's thread pointer
adrp x10, keypage
ldr  x10, [x10, #ctx_slot]    // where nxhost stores our context
ldr  x10, [x9, x10]           // this thread's context
ldr  x9, [x10, #x18]          // the game's x18 value
ldr  x0, [x9, #0x10]          // the original instruction, x18 -> x9
str  x9, [x10, #x18]          // write x18 back
ldp  x9, x10, [sp, #-16]      // restore
b    back                     // and continue after the original

To pick the scratch registers, nxpatch disassembles the instruction with Capstone, gets every register it reads or writes, and takes the first ones that aren't used (x9 to x17 first, since those are temporaries anyway). Renaming x18 itself isn't guessed either : it swaps x18 in the disassembled text, tries re-encoding each register field, and keeps the encoding that disassembles back to exactly the expected text

Then there are all the cases that make this “fun” :

  • instructions that move sp (a push or a pop) : saving scratch registers under sp would get overwritten, so those spill into the thread's context instead
  • PC-relative instructions (adr, adrp, literal loads) : the trampoline lives somewhere else, so their target gets recomputed from their new address
  • branches : conditional branches get retargeted (inside the trampoline or back to the game), and an unconditional branch in the middle of a patched sequence is refused
  • atomics : ldxr / stxr pairs are how arm64 does atomic operations, and jumping out in the middle of one can make it fail forever. So if anything between the two needs patching, the whole sequence gets moved into the trampoline together
  • jump tables : the compiler turns some switch statements into an adr x18 + table lookup + br x18. A trampoline can't know where that jumps… unless it knows the table. So nxpatch walks back from the br x18, finds the table's address, its entry size and the bound check, and lists every possible target. The trampoline then compares the computed address against that list and branches to the right one directly (a few tables the pattern can't find are listed by hand, per module)

A lot can go wrong in there, so nxpatch has to understand what it's looking at, and it refuses to patch when otherwise. An unknown system register, a branch it can't follow, a jump table it can't resolve : it stops and tells you where

svc

System calls are easier. Each svc #n becomes a trampoline that saves x16 and x30, puts n in x16, loads the svc gate address from the keypage, calls it, restores and branches back. The gate lands in nxhost's svc.rs, which reads the number and the arguments from the game's registers and does the job (0x08 creates a thread, 0x09 starts it…)

System registers

  • TPIDR_EL0 (the thread's own pointer) : reads and writes go to the thread's context
  • TPIDRRO_EL0 (the read-only one the kernel sets) : read from the context too, since on Apple platforms it's Apple's
  • CTR_EL0 (cache info) : replaced by the constant a Switch would return
  • CNTPCT_EL0 (the timer) : becomes a read of the virtual counter, scaled by a multiplier from the keypage so it ticks like a Switch's

nxpatch also applies a few per-game patches the same way, each checked against the original bytes first so a wrong game version never gets patched (that's how Mario Kart supports any aspect ratio : its camera stores get a trampoline that scales them). And the trampolines are tested by running them in Unicorn and checking that no other register got touched

Patching a game looks like this :

nxfs --keys prod.keys extract --base base.nsp --update upd.nsp --out work/game
nxpatch rewrite --out work/game/patched work/game/exefs/main work/game/exefs/subsdk...

And because all of this happens before the game runs, the patched code is just… code. On macOS it's loaded at launch, on iOS it gets linked into the app when you build it. That's why there's no JIT anywhere

nxhost : pretending to be a Switch

Running the code is only half of it. The game still expects a whole Switch around it, so nxhost pretends to be Horizon (the Switch's OS) :

  • the kernel (kernel.rs) : threads, memory, sync, all the svcs nxpatch redirected. It even has its own Horizon style scheduler with cores and affinities, because that's what games expect (on iOS we let the system schedule the threads instead, with NX_NO_SCHED=1)
  • the services (services/) : on the Switch, almost everything (files, saves, controllers, audio, network…) is a service the game talks to over IPC. nxhost answers those calls with the Mac/iPhone equivalent (NX_TRACE_IPC=1 logs every one of them)
  • the videos : decoded too, H.264 and WebM (through libvpx)

So yes, technically the OS part is emulated (that's called HLE). But the game itself runs natively, and that's where all the performance comes from

NVN → Metal

The Switch's GPU is an NVIDIA Maxwell, and games draw through NVN, Nintendo's own graphics API. iPhones only speak Metal. This is where most of the work went, it's all in nvn/

Getting the game to call us

Games don't link NVN directly. They call nvnBootstrapLoader to get nvnDeviceGetProcAddress, then ask for every NVN function by name. nxhost answers both, so for each name it decides what the game gets :

  • a Rust implementation : command buffers, draws, textures, memory pools, syncs, presenting…
  • the real NVN driver : the game ships NVN's driver in one of its modules, and it's arm64 code too ! For functions that only fill structs (blend state, depth stencil state, builders' setters and getters…), nxhost just hands back the driver's own function, and it runs natively like the rest. To find them, nxhost knows where the driver's name and function tables are for each driver build
  • a stub : anything else gets logged and returns 0. At exit nxhost prints every NVN function the game called and whether it was real or a stub, which is basically the to-do list for a new game

Same idea for device properties : the ones that describe the Switch's GPU get the values a Switch would give, the rest go to the driver's own helper

Command buffers

NVN works like Metal : you record commands into a command buffer, then submit it. nxhost records the game's commands into its own list (bind this program, these textures, set the viewport, draw…) and replays them into Metal render and compute encoders when the game submits. NVN's states become Metal pipeline and depth stencil states, and the primitives Metal doesn't have (quads, triangle fans, line loops) get rebuilt as plain triangles or lines

Textures

Switch textures live in the game's memory in Maxwell's block linear layout : the image is cut into GOBs (64 bytes wide, 8 rows high, so 512 bytes each), and GOBs are stacked into bigger blocks. Metal wants plain rows, so nxhost reads the texture's format, size, mip levels and block height, works out the exact layout, and untiles it before uploading (tex.rs)

The game can also change a texture from the CPU at any moment without telling anyone, so nxhost keeps hashes of each texture's memory and re-uploads it when they change. And compressed formats the GPU can't sample get decoded on the CPU

Shaders

Shaders are the painful part. Games ship them already compiled for Maxwell : a header (the SPH, which says what stage it is, its inputs and outputs) followed by raw GPU machine code. nxshader turns that into Metal Shading Language :

  1. parse the header
  2. decode every Maxwell instruction (the decoder alone is more than 5000 lines 😭)
  3. write MSL out of them : registers become variables, predicates become bools, constant buffers become buffer reads, the condition flags are kept around for the instructions that use them
  4. compile it with Metal

For control flow we don't try to rebuild ifs and loops at all. The whole shader goes into a loop with a switch on a program counter, and a branch just sets the counter and jumps back to the top :

uint pc = 0u;
for (;;) {
switch (pc) {
    ...
    if (p0) { pc = 12u; continue; }

Metal's compiler is pretty good at cleaning that up, and it means any shader the game has works the same way

All of this is way too slow to do in the middle of a race, so it happens ahead of time : Prepare scans the game's files for NVN shaders (they start with a magic and a valid SPH), translates and compiles every one into Metal libraries, and the iOS build bundles them into the app. At runtime, a shader is looked up by its hash, and only a miss gets translated and compiled on the fly, in the background, which is when you'd see a black frame or two

If you want to poke at it, shdis and shmsl disassemble and translate a shader by hand, NX_DUMP_MSL=1 writes every translated shader out, and echo 1 > work/capture dumps the next frame's draws and render targets

Mush8

Mush8 is where all of this started : Mario Kart 8 Deluxe, natively on Mac and iPhone, at 60fps. On top of the game itself :

  • multiplayer : LAN (a Switch can join !), online through lan-play.com servers and wireless with nearby Mush8 players
  • your iPhone (or any iPhone) can be a controller for your Mac, and any phone's browser can too, no app needed
  • any aspect ratio, with the HUD kept at the edges
  • spatial audio from the game's 5.1 mix
  • RomFS mods

You need an iPhone with at least 6 GB of RAM (8 GB is comfy)

A lot of people helped on this one : codedpalette, hczr-dev (4K and 60fps split screen !), lyccccy, FritteHD, justr4mzi and 184904. Thanks a lot guys !!

Velocity : the second game

Mush8 eventually grew into nxhost, a toolkit that isn't tied to Mario Kart anymore. Velocity is the proof : Sonic Colors: Ultimate, running the same way

Colors Ultimate is a Godot 3 game, so a very different engine. Only three of its modules get loaded : the game itself, the NVN driver and the movie library (NVIDIA's GL driver and the shader compiler stay out). The whole Velocity binary (main.rs) is around 80 lines, it basically tells nxhost where the dump is and what to load :

modules: ["main", "subsdk2", "subsdk3"].map(String::from).to_vec(),
nvn_sdk: Some("subsdk2".into()),

Most of the work went into making NVN → Metal behave with what Godot throws at it (its shaders, its glow, its save files…). And every fix made for Velocity goes into nxhost, so Mush8 gets it too, and the other way around

Porting your own game

A port is basically three things :

  1. extract and patch the game's code once, with nxfs and nxpatch
  2. a small binary that starts nxhost with your dump and the modules to load
  3. fixing everything your game does that the previous ones didn't (that's the fun part)

The nxhost README has the details, including the iOS part

Can I try it ?

Yes ! everything is open source : Mush8, Velocity and nxhost

No game files are included tho (Nintendo ninjas are always watching), so you need your own dump : prod.keys, the game, its update and the firmware from your Switch. Build instructions are in each README

People also asked about Android : technically we could, by using Vulkan instead of Metal

That's basically it ! Contributions are welcome, and if you really want to support this there's my ko-fi (it'll stay free and open source either way)

All posts