︎
Work                  Events                 About

RIVEN RATANAVANH



Making Visual Art with GANs

Week 2

Notes from Class
Better datasets = (usually) better results.
(Dataset creation isn’t what data scientists usually focus on).

We tried playing with Aphantasia.

Types of GANs we’ll be working with:

Text to Image

Style Transfer - takes ‘texture’ from one image and applies to the ‘content’ of another image (almost like applying texture in 3D modelling). Lots of power, but was... ruined by starry night selfie style transfer apps.

Pix 2 Pix - datasets have to be manually categorized.
Self-driving cars happen to be trained on this.

CycleGAN/MUNIT - matches unpaired images and generalises translation. e.g. horse to zebra

StyleGAN - not image input, but vector (512)
High level of control, can modify features, can generate ‘fake’ images.

***


Homework

Explore pre-trained models in Co-lab. Note:
What model you used
What the inputs and outputs were
What questions you have about the model


Model 1
First Order Motion Model for Image Animation


Input 1

Input 2 (video)

( from the colab notebook)


Output


Used the vox option.

Input 1

Input 2 (video)

Output

For this I used the namu option.

What happens if I don’t put a facelike photo in as the photo input?




Model 2
Infinite Nature

Input:


Output:



Input:


Output:


Input:



Output:


Input:



Output:



Model 3
Image generation

I didn’t know how to modify the inputs in this notebook, but this was the output the first time I ran it:

The second time I ran it:



Model 4
StyleGAN2

Probably what I was most interested in because of this.

I didn’t really understand how to modify the inputs here - need time to explore further.




Overall, I discovered it’s fun to give models slightly different inputs than they were designed for.


*


Models that didn’t work that I’m interested in:

1. Wanted to try SPADE: image synthesis from semantic layouts but when I tried to open it in colab I received a message saying the notebook was not found.

2. Style Transfer
Ran into this error.



3. 3D Ken Burns (from this article)
Ran into two errors:

I followed this advice and entered a new line of code to install chainer, which seemed to solve it.



But still ran into this error:



One more of interest I didn’t have time to try was 3D Photography.



Not for this course, but I found the sample of music made by Magenta in this article really compelling.