Project Proposal – Logan Sullivan

While I won’t be using the article much for information going forwards with creating my project, Chic-Marker by Rong-Hao Liang et al got me to consider exploring 3D-printed fiducial markers, ultimately inspiring my project idea.

From Chic-Marker

For those unaware, fiducial markers are small grids of high-contrast pixels, usually black and white, which form clear and identifiable patterns for cameras and software to spot and identify. QR codes are a highly specific, much more detailed type of fiducial marker (FM), containing self-anchoring patterns and a bunch of raw binary data. In more general contexts however, FMs usually consist of much simpler patterns, where each pattern corresponds to a unique piece of information established ahead of time, like a dictionary. They can be used for identifying special objects, indicating faces on a cube, marking locations, identifying different actors or items, and much more.

While thinking about something interesting to make with physical, 3D-printed fiducial markers, my mind eventually drifted to FEZ, an indie puzzle-platformer game from 2012. It’s a phenomenal game (if you’ve ever enjoyed TUNIC, The Witness, or Animal Well, you’ll love this too), and the idea I have is unfortunately a big spoiler for one of the game’s deeper puzzle systems. So scroll on down if you know all about FEZ’s in-game language, or if FEZ doesn’t sound like your cup of tea.

(spoiler block)



























(/spoiler block)

In FEZ, several puzzles and almost all non-mandatory dialogue use a special language created by the game, called Zuish. It uses English as a base, but replaces all the letters with new symbols and requires you to tilt your head 90 degrees clockwise to read normally. One room in the game contains a pillar with a pangram that reads “the quick brown fox jumps over the lazy dog” along with an animated sprite of that very sentence taking place nearby as a hint.

From the FEZ wiki

The language also contains numerals 0 through 10, taught in a much more complicated way elsewhere in another room. The details aren’t too important for now, however.

So why am I talking about FEZ all of a sudden? Well, if you take a close look at the Zuish letters, you may notice something familiar…
They’re all small, square grids of pixels with two colors! They’re practically begging to be used as fiducial markers!

Translating Zuish messages typically requires you to draw things out with pen and paper or use a website, but what if there was a more hands-on approach? Something more tactile, easy to manipulate?

My project proposal is to do just that, combine 3D printing and fiducial markers to physicalize the Zuish language and make it automatically translatable via taking a photo and passing it to a simple program.

To be more precise, my plan is to have large baseplates which can interlock to form as large of a writing space as desired, and contain slots for letter pieces to fit in. By pausing prints partway through and changing out the filament, I can apply the FM patterns as part of the printing process. Small magnets could be used to help pieces stay in place, with only a shallow height difference for the slots to maintain alignment without making it hard to remove pieces. I would then create a simple website which scans an image to translate from Zuish, and can take a text prompt to generate an image for the Zuish counterpart.

However, you may have noticed that many of the letters in Zuish are the same as each other, only different by rotations of 90 degree multiples. As such, the baseplates will need special FMs in the corners to orient everything. Also, some letters share the same exact symbol, rotation and all. This goes for the pairs K & Q and U & V. While it rarely ever causes problems in normal sentences, I’d still like to remove this ambiguity, without making new symbols entirely. As such, I’m going to examine a paper by Douglas Tybusch et al where fiducial markers of more than two colors are examined. With such a system in place, I could have separate colors for these entwined pairs to show ultimate clarity. The website could even have a function for taking an image (either of the physical Zuish pieces or of a digital image generated from English text) and export it with the pair colors being converted to standard black and white to match the Language’s normal appearance.

A bonus result from physicalizing Zuish is that instead of needing to tilt your head to read it, you could rotate the board itself! As such, the website should also allow for scanning and generating in post-rotated form, for easier reading and writing of the language.

An app which could perform the scanning in real time without needing to save the photo would be a nice stretch goal, though I use an iPhone, meaning I would need Apple’s permission do get anything done, as opposed to much easier and more independent methods on Android devices. As such, I will likely stick to just a mobile-accessible website for now.

Deliverables

• A set of small 3D-printed tiles to represent the letters of the Zuish alphabet from FEZ
  • There should be multiple copies for each letter
  • The duplicate symbols in the letter pairs K & Q and U & V should be differentiated by different filament colors
• A set of baseplates which have slots arranged in a grid to put letter tiles into
  • Baseplates should be able to connect together to create an arbitrarily large grid
  • Baseplates should be designed such that the translator script can identify them together as one collective grid
  • The number of slots per baseplate and number of baseplates printed should be enough to write at least 1-2 simple, medium-length sentences in one grid.
• A javascript program for translating Zuish by interacting with the 3D-printed tiles
  • Given an input image of the grid-placed tiles, output the English translation as text
  • Given an input string of English text and other parameters such as max column height, output an image representing the Zuish translation
  • If feasible, make the program accessible in-browser through a simplistic webpage, for both desktop and mobile devices

Timeline

• Week 1:
  • A small printed sample of the baseplates and tiles, enough to demonstrate the symbols and connectable baseplates
• Week 2:
  • Prototype translation program which can output Zuish images from English text, and progress on Zuish images to English text
• Week 3:
  • Refined baseplate and tile models printed in full quantities, and a completed translation program

Related Works:

• Chic-Marker: Fashionably Fusing Fiducial Markers into Apparel and Accessories
• Color-Based and Recursive Fiducial Marker for Augmented Reality