Using Blender with openMosix
What you will need for this bite-sized example
- 2 (or more) networked machines running dyne:bolic 1.3
- storage space for the blender models, renders, etc.
- a .blend scene ready to batch render
- Marc O. Gloor's render script
In addition to the obvious solution of burning multiple CD's, you can run d:b from a hard drive by 'docking' dynebolic. Simply copy the 'dyne' directory from the CD to the root directory of a readable hard drive partition. Only read access is needed, so any filesystem d:b can read will work. For example, on a Windows machine, copy the dyne directory to C:, creating a C:\dyne. In a linux environment copy straight to /, creating a /dyne. Now reboot with the d:b CD and it will locate the new docked install and eject the CD. Repeat the process on each machine that will form the cluster. Assuming the machines are connected to the same network (the faster the better) the cluster will be configured by the time you finish booting. Remember, too, that you can add nodes at any time, just by booting another d:b on the network.
You will need to have at least 60 megabytes of free space for the scene elements, rendered animation frames and a few other odds and ends. In d:b, the local hard drive partitions are known as /vol/hd#, where # is the number representing the order it which it was encountered. For example /vol/hd1 would be d:b's name for the typical Windows C: drive, or /dev/hda1 under most Linux flavors.
A blender sceen to render
We'll use the time honored blacksmith scene from download.blender.org . Pick the .tgz version. Explode the downloaded file with:
tar xvzf blacksmith.tgz
This will create a 'blacksmith' directory. We will need to make a few minor tweaks to set things up to render frames as expected for this example. Inside the blacksmith directory, create a 'rendered' directory. This will be used to hold the animation frames we are going to render. Now, launch Blender and load the blacksmith.blend. Make sure you have scene 04_06 selected. Bring up the render settings panel to set these options:
- frame size, pick 640x480
- file type, pick jpeg
- output directory, our 'rendered'
- turn on file extentions
This screenshot highlights the relevant changes. Check out one of the excellent Blender tutorials on the web to build an understand of all of these options. Save your work as a new .blend file, so you can go back to the original if needed. Call it blacksmith-jpeg.blend. You can exit Blender at this point.
The render script
Blender needs a little bit of coaxing to use the cluster resources. OpenMosix works by moving processes from a heavily loaded machine to a more lightly loaded machine. Blender runs as a single process, so openMosix can't do much, other than move the one process to another machine. If you split the work to be done into 2 or more process, openMosix will handle the rest. You should split the work into at least as many processes as you have nodes on the network. You could split the workload manually, but for this example, we will use a pre-made script to do the splitting. Download Marc O. Gloor's excellent render script (local mirror - render.gz). Ungzip the script and make it executable with:
chmod +x render
You can store this script anywhere, but If you copy it into your 'blacksmith' directory, it will be in the search path for this example, since by default in dyne:bolic's zsh shell the current directory is in the path.
There are a lot of options available in openMosix but dyne:bolic is preconfigured to use auto-discovery, so you don't need to do much. From an xterm, cd to /usr/mosix and these commands are available:
This will verify that you have a cluster, and show the nodes that are available. Here is a sample output from a 4 node system:
[d:b] /usr/mosix #openmosix status openMosix: No map-file found. openMosix: Falling back to autodiscovery mode using /usr/mosix/sbin/omdiscd This is OpenMosix node #261 Network protocol: 2 (AF_INET) OpenMosix range 261-261 begins at 10.0.1.5 OpenMosix range 407-407 begins at 10.0.1.151 OpenMosix range 266-266 begins at 10.0.1.10 OpenMosix range 258-258 begins at 10.0.1.2 Total configured: 4
There are other useful options. Display a complete list by executing
openmosix with no arguments.
mtop is an openMosix aware top command. Note the N# column indicating what node the process is running on.
10:37pm up 21:50, 0 users, load average: 0.32, 0.15, 0.05 45 processes: 44 sleeping, 1 running, 0 zombie, 0 stopped CPU states: 160.8% user, 7.1% system, 0.0% nice, 0.0% idle Mem: 254936K av, 167160K used, 87776K free, 0K shrd, 17812K buff Swap: 642560K av, 5344K used, 637216K free 83440K cached PID USER PRI NI SIZE RSS SHARE STAT N# %CPU %MEM TIME COMMAND 1169 root 17 0 39328 38M 676 S 407 85.5 15.4 0:17 blender 1172 root 14 0 37700 36M 1624 S 266 42.6 14.7 0:11 blender 1166 root 18 0 42436 41M 1656 S 266 38.3 16.6 0:11 blender 1175 root 14 0 26592 25M 3408 S 0 0.7 10.4 0:01 blender 1181 root 18 0 848 848 692 R 0 0.3 0.3 0:00 mtop 1098 root 10 0 2208 2208 1576 S 0 0.1 0.8 0:24 sshd 1 root 9 0 476 476 420 S 0 0.0 0.1 0:06 init ...
Just as mtop is to top, so is mps to ps.
[d:b] /usr/mosix #bin/mps PID TTY NODE STAT TIME COMMAND 1100 ? 0 S 0:00 -zsh 1146 ? 0 S 0:00 xterm 1147 ? 0 S 0:00 zsh 1159 ? 0 S 0:00 /bin/sh render blacksmith-jpeg.blend 1 235 4 1166 ? 266 S 2:40 (blender) 1169 ? 266 S 5:51 (blender) 1172 ? 258 S 4:16 (blender) 1175 ? 258 S 1:48 (blender) 1199 ? 0 R 0:00 bin/mps
It is interesting to note in this display that openMosix has moved 4 jobs to just 2 nodes, and the node that started the processes is not running any of the jobs. The machines on the cluster were a mixture of older slow machines, and newer faster machines. openMosix handles balancing the load it is given. In cases like this, you might squeeze more out of your cluster by increasing the number of jobs running.
mosctrl offers even more detail information and control for your cluster.
[d:b] /usr/mosix #bin/mosctl Usage: bin/mosctl command Available commands: stay/nostay, lstay/nolstay, block/noblock, quiet/noquiet, mfs/nomfs, expel, bring, gettune, getdecay, whois [mosix_no|IP-address|hostname], getload [node-number], getspeed [node-number], status [node-number], getmem [node-number], getfree [node-number], getutil [node-number], getyard, setyard [node-type], setspeed speed, setdecay interval(seconds) slow(/1000) fast(/1000), isup [node-number] zsh: 1208 exit 1 bin/mosctl [d:b] /usr/mosix #bin/mosctl getmem 266 free memory = 217960448 of 234815488 [d:b] /usr/mosix #bin/mosctl getmem 261 free memory = 247595008 of 268423168 [d:b] /usr/mosix #bin/mosctl getspeed 266 speed=26842. [d:b] /usr/mosix #bin/mosctl getspeed 261 speed=8240.
The last two commands shown offer a clue why node 266 gets more work than 261.
Ready to Render
In an xterm, cd to your 'blacksmith' directory. Issue the render command as shown. Change the final 4 to match the number of nodes in your cluster.
[d:b] ../blacksmith #render blacksmith-jpeg.blend 1 235 4 Rendering 234 images from blacksmith-jpeg.blend on 4 nodes. Tasks forked, network rendering in progress. Job started at: 10-04-04 22:35:46 Please wait while rendering...
Now comes the waiting, which will depend on the number of nodes and the speed and memory of each node. While the jobs run, try out the mtop, mps and mosctrl commands in another xterm to get the feel for what is happening and what the capabilities of your cluster are. Eventually you will be presented with the completion message.
Rendering successfully finished. Job ended at: 10-04-04 22:56:15
Follow up with a
ls rendered to list your output files. Assuming you find 230 some .jpg files, congratulations on your cluster render!
Building a movie from jpeg files
Now that everything is rendered, you might want to see this animation, right? d:b has the tools for that, too. Just cd into your rendered directory and issue this:
mencoder -mf on:w=640:h=480:fps=12 -ovc copy -o output.avi \*.jpg
In just a few moments, you should have an 'output.avi' file, ready to play. For more info on this, and similar capabilities in mencoder check in the MPlayer manual under Encoding with MEncoder .
Now you've got power. Go create something wonderful!