For this project I wanted to scratch a new itch, which is working with shaders.
It has always been on my mind as something to investigate and learn about, though I had zero knowledge of it prior to starting this project.
Without knowing anything, I knew I wanted to try making an image of spherical and give it texture and dimensionality, something that looks like a moon. It doesn’t have to be interactive, but if I can achieve it, I want it to move or change over time.
I found this resource https://itp-xstory.github.io/p5js-shaders/# and this https://thebookofshaders.com.
A lot of the below is the ITP resource above regurgitated in my own words which helped me sort and understand the information.
What are Fragment Shaders?
Drawing shapes in function draw() uses the CPU. The CPU does every calculation (every pixel at a time, as we have learned in the pixels unit). So more complex drawings slow down the program.
In contrast, the GPU is capable of parallel processing. Using a shader, everything can be drawn at once.
Anatomy of a Shader
Shader files are written in GLSL (OpenGL Shading Language). It is a lower level language than JavaScript because it deals directly with the GPU.
Shader language is “strongly typed” which means that unlike in p5 where we can say ‘let’ or ‘var’ and JavaScript will figure out what kind of variable it is, GLSL requires us to define the type of all our variables. These variables can include float, int, vec2, vec3, vec4. Shaders use vectors to store not just direction or position, but also color.
The .vert file handles all of the geometry on the canvas.
The .frag file handles coloring in pixels.
The vert file runs first, then then frag file runs and colors in geometry that is passed from the vert file.
.vert Files
Each .vert file needs to contain these preprocessor directives:
#ifdef GL_ES
precision mediump float;
#endif
And this:
attribute vec3 aPosition;
Everything in the shader .vert file is run for each pixel in the canvas.
Specific to p5, in the main() function of a .vert file, there must also be a line to scale this aPosition due to a bug.
(I find this really funny.)
Basically what needs to happen to fix this is that the position information in vec3 with 3 values is copied into a vec4 with 4 values:
vec4 positionVec4 = vec4(aPosition, 1.0);
1.0 here indicates that that is what goes in the w parameter. According to standard vector math, w = 1.0 indicates the vector is a position whereas w = 0.0 indicates the vector is a direction).
However we are not using the z value as we only need x and y for the canvas, so we use .xy after the vector name to indicate that. Then to correct for the bug we scale it by two and move it down and left on the screen:
positionVec4.xy = positionVec4.xy * 2.0 - 1.0;
The main function then ends with this line:
gl_Position = positionVec4;
Each .vert file needs gl_position in order to send the above scaled position information from vert to the fragment shader in the .frag file.
.frag Files
Also requires the same definition in the header as .vert files do.
In the main() function, a solid color can be defined as such:
vec3 color = vec3(0.0, 0.0, 1.0);
vec3 takes 3 arguments for RGB, but unlike in a p5 sketch values range from 0.0-1.0 rather than 0-255.
Then gl_FragColor tells the GPU what color the pixel is. It takes 4 arguments as vec4, and the values are R, G, B, and Alpha.
gl_FragColor = vec4(color, 1.0);
‘Swizzling’
We can access the xyzw or rgba values in a vector by ‘swizzling’. GLSL doesn’t know whether the vector components denote position or color, but it’s good practice to use the appropriate letters.
e.g. accessing colors
vec3 color = vec3(0.0, 0.0, 1.0) // Blue
vec 3 colorCopy = vec3.rgb
Using Swizzling to change colors
vec3 color = vec3(0.0, 0.5, 1.0);
color.r = 0.833;
vec3 newColor = color; // Gives us R= 0.833, G = 0.5, B = 1.0
Functions
Variable qualifiers (come before declaring variable type and) include:
-
Const - declares a variable that stays constant throughout the program
-
#define - runs script before shader compiles
-
attribute - used to receive vertex data from p5 and convert to a form readable to .frag files, only used in
-
uniform - constant variables that stay constant per
- varying - parameters that are specific to pixel. Typically used to pass texture coordinates.
(got it)
I discovered that either way of entering values creates the same result:
This is the sketch.
I simply replaced u_resolution with u_mouse to make this interactive sketch. Kind of reminiscent of risograph printing.
The Actual Project...
Now looking at the Book of Shaders, I wanted to use this knowledge to create the sort of moon like shape using shaders. I was interested in Fractal Brownian Motion and wanted to apply that to a shape.
I made this using the first example on that page and what I had learned about applying shaders to a shape from here.
I used this to round out the cornder of the ellipse:
I also made this by modifying the second bit of example code.
Thoughts
I set out on this project not knowing what I didn’t know. I thought that I was going to make something relatively visually complex but that the coding would be relatively straightforward. I assumed that learning to work with shaders would be as straightforward as learning to work with a new p5 library. What I didn’t know was that to work with shaders I had to learn a new language based on C rather than JavaScript.
While I had hoped to code a shader from scratch that would be able to achieve what I needed, I realized that to do that was far more difficult that I had expected (and not entirely necessary).
I do finally feel like I’m able to see the world through code. I’ve experience similar with design, movement, and figurative drawing. Perhaps the most illustrative example here is drawing -- when do enough of it and I feel like I walk about seeing the 3D world as contours of shapes or geometry automatically flattened into 2D. Now I feel like I’m seeing the world as pixels, or... after this week, textures stretched over 3D shapes much in the way shaders work.
At the beginning of the semester, I thought that I would perhaps want to use code to create sketches that can be projected onto spaces or things. That remains true, and I’ve realized that the reason why I gravitate to projection despite it being a pain to work with is that it is a way to bring these sketches into physical spaces. While my desire to use projections to transform objects and spaces remains true, what become clear to me is that I am not interested in using projections to create more flat things, to directly project 2D sketches onto flat objects.
ICM has empowered me to sustainably keep learning code, and if there’s anything I would have known to ask of my first semester at ITP, it would have been that.