The goal of my project was to create a set of physical baseplates and tiles which connect together to write phrases in Zuish, the in-universe language from the game FEZ. Additionally, it was my goal to create a program which can take images of the physical tiles and translate them into English.
The principle idea behind the project is creating a set of physical, rearrangeable fiducial markers, and a scanner to process them. Since Zuish only uses 24 unique symbols to represent all 26 letters, I wanted to also remove ambiguity in the letters that have to share a symbol, by using additional colors rather than creating new symbols.
Background Research
The project which originally got me interested in the idea of physically-printed fiducial markers is Chic-Marker, where markers were integrated into clothing, bags, and other textiles, while still maintaining a (mostly) fashionable appearance.
From Chic-Marker
Before diving head-first into my project idea, however, I also wanted to see how much of a precedence there is for fiducial marker systems that use more than two colors. As it turns out, most FM systems still use binary colors, but it’s certainly not impossible or unreasonable, as seen with a 2017 IEEE paper led by Douglas Tybush. While I ended up taking a different approach, it was good to know that multi-color markers aren’t outlandish.
Another interesting project I found was DynaTags, a low-cost system of layered sheets of paper which can be manipulated to form a variety of symbols with one marker, instead of being stuck with static, unchanging markers.
Process
In order to use multiple filaments and control when they get changed out, I needed to prepare my prints with a mix of slicer software and custom-generated gcode. The main shape of each model was created in Rhino using Grasshopper, and would get passed into Cura to be sliced. Then, I would examine the resulting gcode files, adjusting the endings and finding the last position of the print head. With this info, I could then generate the custom gcode with another script, and add it to the files.
The gcode script could very well have been made in Grasshopper, but I’m more experienced with Lua, and I didn’t want to deal with file I/O or other shenanigans in Grasshopper, so I made a separate program written in Lua. Depending on what I’m printing out and what parameters I use (number of tiles, symbols to use, etc), the Grasshopper script would generate instructions for the Lua script, telling it what symbols to draw and where.
The physical components consist of baseplates with a grid of slots for tiles and edges that connect together like puzzle pieces, tiles with Zuish symbols on top and magnets below, and physical translation keys for reading and writing messages by hand.
After enough time working away at the code, and several test runs of the tiles, I eventually got things how I wanted them and with a high degree of consistency. There weren’t any hugely notable issues I can think of, just gradually tinkering until everything worked. From there, I simply had to print everything out, which took about four days of non-stop printing, support removal, magnet gluing, and assorted cleanup.
In the end, I printed 4 baseplates, 160 tiles, 2 physical keys, and I used 416 magnets and over 820 grams of filament.
As for the scanner/translator, I used js-aruco, a Javascript port of ArUco as the baseline for my fiducial marker detector. I had to heavily rework how it differentiated between symbols, as normally Zuish symbols are not possible with ArUco’s row palette. I also had to allow for detection of red vs blue vs white colors to handle the shared symbols, and other assorted tweaks to the library to tune it to my needs.
Then, I used HTML and Javascript to create a github pages website for the scanner, handling all the logic and so on to actually detect the baseplates, put the markers in order, and translate them. It has options for using your webcam (or phone camera), uploading images, or giving English text to turn into Zuish images.
Outcome
I’ve documented all the steps needed to print out parts of my project in this repo, and the scanner can be found here.
Future Work and Reflection
This was a really fun project, giving me an opportunity to make something related to a great puzzle game I hold near and dear to my heart. Getting everything to work together was like a puzzle in of itself, making sure each program worked in harmony despite my convoluted workflow, and fitting as many tiles into one baseplate as possible without making them too small to work with. Not to mention actually handling the custom gcode generator and the scanner, as they went through several micro-iterations as I came up with better and better ways to execute them.
Since I designed the baseplates to be modular, I can always print out more if I need more space to write, and same for the tiles if I run out of certain letters. There’s not a lot I feel like needs improving with the physical models, though it would be nice to some day tweak the scanner to let it translate screenshots from FEZ. The tricky part is that the game doesn’t have special corner markers like my baseplates do, meaning it’s harder to get a 100% reliable reference for the spacing between symbols, which is needed for detecting space characters. It’s certainly not out of the realm of possibility, however.