Inteligent paint .

Overview :


I show how to teach machines to paint like human painters, who can use a few strokes to create fantastic paintings. By combining the neural renderer and model-based Deep Reinforcement Learning (DRL), our agent can decompose texture-rich images into strokes and make long-term plans. For each stroke, the agent directly determines the position and color of the stroke. Excellent visual effect can be achieved using hundreds of strokes. The training process does not require experience of human painting or stroke tracking data.









Installation


Use anaconda to manage environment

$ conda create -n py36 python=3.6
$ source activate py36
$ git clone https://github.com/tajalagawani/LearningToPaint.git
$ cd LearningToPaint


Dependencies


PyTorch 0.4.1tensorboardXopencv-python 3.4.0

pip3 install torch==0.4.1
pip3 install tensorboardX
pip3 install opencv-python


Testing


Make sure there are renderer.pkl and actor.pkl before testing.

You can download a trained neural renderer and a CelebA actor for test: renderer.pkl and actor.pkl


$ wget "https://drive.google.com/uc?export=download&id=1-7dVdjCIZIxh8hHJnGTK-RA1-jL1tor4" -O renderer.pkl
$ wget "https://drive.google.com/uc?export=download&id=1a3vpKgjCVXHON4P7wodqhCgCMPgg1KeR" -O actor.pkl
$ python3 baseline/test.py --max_step=100 --actor=actor.pkl --renderer=renderer.pkl --img=image/test.png --divide=4
$ ffmpeg -r 10 -f image2 -i output/generated%d.png -s 512x512 -c:v libx264 -pix_fmt yuv420p video.mp4 -q:v 0 -q:a 0
(make a painting process video)


i also provide with some other neural renderers and agents, you can use them instead of renderer.pkl to train the agent:

triangle.pkl --- actor_triangle.pkl;

round.pkl --- actor_round.pkl;

bezierwotrans.pkl



Training


Datasets

Download the CelebA dataset and put the aligned images in data/img_align_celeba/******.jpg


Neural Renderer

To create a differentiable painting environment, we need train the neural renderer firstly.

$ python3 baseline/train_renderer.py
$ tensorboard --logdir train_log --port=6006
(The training process will be shown at http://127.0.0.1:6006)

Paint Agent


After the neural renderer looks good enough, we can begin training the agent.

$ python3 baseline/train.py --max_step=200 --debug --batch_size=96
(A step contains 5 strokes in default.)
$ tensorboard --logdir train_log --port=6006

References


Huang, Zhewei and Heng, Wen and Zhou, Shuchang .


Learning to Paint with Model-based Deep Reinforcement Learning

Zhewei Huang∗ Peking University

hzwer@pku.edu.cn

Wen Heng Megvii Inc

(Face++)

hengwen@megvii.com

Shuchang

Zhou Megvii Inc (Face++) zsc@megvii.com .


0 views
CXS-White-Base-01.png
Official
incubator

Coordination : Hassan Jacobsen