The following is a list of writings, technologies, and videos that have changed the shape of how I approach and think about design in a significant way over the last few years. Each concept or piece of technology is something I recommend designers have at least a cursory understanding of. While they are all currently being explored, I still view their potential as largely untapped in digital ui design.

Generative Design

Generative design output. Source: Arup

Generative design can be utilized to spend more time curating and reviewing designs than manually constructing them for review. Adding constraints to generative algorithms can allow for the automated generation of solutions that might not be quickly intuited or explored due to the sheer manual labor to construct them all. Curating options at the scale generative design allows for can help you more rapidly understand relationships between values.

Generative vs. Possibility Space
Sampling
Design method for critical structural steel elements
Project Dreamcatcher - Autodesk

Computational Design

Within design - what can we compute? What can we not? What are the physics of user interfaces?

Computational Design - Article
Codify: Parametric and Computational Design in Landscape Architecture
“The business sector is littered with companies that took risks and failed, but also others that did not innovate (or innovated too late or even too soon).”

“What we yearn for as a profession is computation. That concept goes far beyond the tool kit. Computational design is the systematic method for critical thinking that emphasizes thought process and iteration over memorization and duplication. It stresses the linking of ideas, and the interaction between the parts of the problem and the solution.”

“The transition will be a complex evolution from “static” built/urban environments to “dynamic” self-constructing, living, breathing, and even artificially intelligent (thinking) environments.”

“In my opinion, and this is personal, there is no other way to work than computationally. If I was starting a practice in engineering right now, I would be starting out to set the curve, rather than follow the curve. TO do that, you have to arm yourself with the best computational tools because if you don’t start there, in my opinion, you will not have a difference to show.”

Computational Design - Book
“Parametric or Generative design has the potential to overcome the mass-produced, homogenous, and disorienting sterility of 20th century architecture. It has the potential to re-associate with historic practice, and amplifying assimilated knowledge. ”

“Indeed, if all the potential solutions are already out there and it is simply a question of searching for them, it potentially undermines the whole notion of the ‘design genius’. Further, if there is any creativity in that process it lies surely in the creative us of such search engines. The designer emerges less as the demiurgic top down controller and more as someone who harnesses the productive capacity of informational processes. As I have noted elsewhere: ‘Of the many potentialities afforded by the the computer, one of the most significant is its capacity to poperate as a search engine. If, then, we think through the logic of search in the context of ‘design’, what such an approach suggests is that if all possible solutions already exist, it is simply a quest of defining a set of constraints and conducting a search, and then selecting on of the many outcomes. The potential implications of this are far reaching.”

Shaders

If you only learn about one thing - make sure it’s shaders.

Beginners Guide to Coding Graphics Shaders
“A shader’s sole purpose is to return four numbers: r, g, b,and a. That’s all it ever does or can do.”

The Book of Shaders
“This book is about the revolutionary computational technique, fragment shaders, that is taking digitally generated images to the next level. You can think of it as the equivalent of Gutenberg’s press for graphics.”

Procedural shader example
Decoding procedural shader
Visual Shaders
How You Can Use AI, AR, and WebGL Shaders to Assist the Visually Impaired

Language Models

GPT-2: Better language Models and Their Implications
“GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text…On language tasks like question answering, reading comprehension, summarization, and translation, GPT-2 begins to learn these tasks from the raw text, using no task-specific training data.”

T5

Tabnine
“Deep TabNine is trained on around 2 million files from GitHub. During training, its goal is to predict each token given the tokens that come before it. To achieve this goal, it learns complex behaviour, such as type inference in dynamically typed languages.”

Wave Function Collapse

Largely used in game design, WFC is a powerful component of constraint based procedural design. Can be used to design everything from poems to 3D worlds.

The Wavefunction Collapse Algorithm explained very clearly
Generating Worlds With Wave Function Collapse
""Using WFC (specifically the Overlap model) makes this sort of generation intuitive, you draw an example and get infinite variations of it. There’s also an aspect of collaboration with the machine, as the resulting output can be surprising and sometimes lead you to change your design.”

WFC Algorithm on GitHub
Bad North + WFC
Interactive demo

Generative Adversarial Networks (GANs)

“The most important one, in my opinion, is adversarial training (also called GAN for Generative Adversarial Networks).This, and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion.” <small style={{ display: ‘block’}}>– Yann LeCun, Director of AI Research at Facebook and Professor at NYU

Wikipedia definition
“The generative network generates candidates while the discriminative network evaluates them.”

What is CycleGAN
CycleGAN
SinGAN: Learning a Generative Model from a Single Natural Image
3D Photography using Context-aware Layered Depth Inpainting

Deep / Reinforcement Learning

There is a certain type of creativity that emerges when computers can attempt millions of attempts at optimizing towards a goal within a given set of constraints. Shown below is a crane that’s had it’s gripping function disabled - learning to pick up a cube from a platform.

Surprising Creativity of AI
Multi-agent Reinforcement Learning
How to teach AI to play Games: Deep Reinforcement Learning