Mario Maker 2 API

Created Saturday, Jun 4, 2022

Over the last few years this site has been mostly dormant, but I recently got inspired to put this website, and specifically the server running it to good use. I chose my webhost specifically so I could have root access at the command line, and I now take full advantage of that because of my most recent project.

Mario Maker 2 is a game I've played for years, since launch, and I've always been interested in putting my skills in programming to the test here. I've finally done just that. Using the incredible NintendoClients made by Kinnay, I, and individuals from the Ninji Speedrunning discord, 0% discord, and Kinnay himself, we now have an entirely free and public way of accessing data previously only available to the switch itself.

Nintendo Switch Online games tend to use a specific library with a specific protocol to communicate with Nintendo's servers. Since the WiiU, this protocol has been NEX. NEX, partially due to the incredible success of many of the games using it like Mario Kart 8 and Splatoon 2, has been extensively documented, to the point where it is fully possible to spoof both the client and server. The Pretendo project has been working on a WiiU NEX server for years, and alongside it NintendoClients, a NEX client, has been becoming more stable and safe to run on your personal account.

In October 2021 I finally took the plunge. After watching the project for years, Tatiaus, a talented kaizo player, contacted me about whether this API I hinted at months ago even existed; that gave me the push to continue and accelerate development. On the 13th, I released the first build, publicly available with no login required.

First release on discord

First, the Ninji server used it to round out their leaderboards, making them 100% accurate for the first time ever. The features I considered most important, the level_info and level_data endpoints, went unused. Not many people outside the Ninji Speedrunning discord knew this existed. Slowly, the API got around, but it stayed a novelty. One of the funniest misconceptions I saw was the guess that my switch was running 24/7 and downloading levels for me. The truth is that this was running on a linux server, using a complete recreation of the NEX client. No switch was involved, except for the various credentials the NEX protocol needs to authenticate with the server.

In November a kaizo player well known for performing incredibly difficult "trump jumps", jumps in the game which were pixel perfect, came to me about using the API for a level viewer he had designed. In time Ji Xiaomai released his viewer, and that was the spark that ignited the entire community. Suddenly now anyone could view any level without the game and even without a switch. Tricky levels that were only possible due to information only the creator could know were exposed. Suddenly now no level could hide secrets.

After the release of this viewer, I ported it to C++ and made it portable enough to run on the browser, so now it wasn't even neccesary to download the app to your PC. With time came new incredible viewers, including one that had animated textures. The count of level viewers was now 5, including the continuation of an old level viewer by streamer and programmer Shoujo and one made from the ground up by Ingram. The community had collectively come together to tear the game apart, utterly destroying the roughly 1 level viewer made for the first game. And in another impressive effort, almost every field of every endpoint was determined, so now stats previously thought impossible to obtain, like the boos on a level or the k/d ratio of a player, were accessible. All this hype has generated an impressive amount of traffic as well:

Web traffic after release

Through all this, the API has stayed public and free. Information should always be as accessible as possible and designed to be as easy to parse as possible by people and computers alike. While I am putting my personal Nintendo Switch Online account on the line, this tool is invaluable to research about the game. So I can proudly say, while Youtube made dislikes private, we made Mario Maker 2 dislikes public.

The logical conclusion to this venture is to pull as much data as we can. So I did the only reasonable thing, went out and bought a 14 terabyte hard drive in order to download Mario Maker 2's entire database. Now it might seem impossible to simply download everything, we can request up to 300 random levels at a time through endpoints like search_endless_mode, but statistically we will recieve less and less unique items over time, almost guaranteed to take years to download 100% of available levels. Nintendo, however, made this easy for us:

def course_id_to_dataid(id):
	course_id = id[::-1]
	charset = "0123456789BCDFGHJKLMNPQRSTVWXY"
	number = 0
	for char in course_id:
		number = number * 30 + charset.index(char)
	left_side = number
	left_side = left_side << 34
	left_side_replace_mask = 0b1111111111110000000000000000000000000000000000
	number = number ^ ((number ^ left_side) & left_side_replace_mask)
	number = number >> 14
	number = number ^ 0b00010110100000001110000001111100
	return number

Course ids are constructed from a regular algorithm from a consistent incrementing index called a data_id. Internally these ids are used to request many known levels, like a list of your own uploaded levels. Because these data ids increment regularly and we know both the first data id, 3 million, and the last through a call to search_new, we can download every uploaded level by sweeping through all 31 million data ids currently allocated. Because users follow similar rules, downloading every user account is also possible.

Running a python program for 4 straight weeks to generate a 2 terabyte database may seem foolish to most, but I'm not putting this perfectly good PC to waste by turning it off. Instead, I did it, ran my computer for all of those 4 weeks, and the resulting data will be a goldmine into how players interact with levels and each other.

In an update post I'll cover the data analysis I do on this massive archive, and release it publicly. Make sure to look out for that!