Google+ Followers

Thursday, July 17, 2014

Ink in Water

One way to achieve this effect is using a looot of particles advected by a velocity field generated by some gas simulation. Why not use the gas sim itself ? Cause in order to achieve the same detail offered by particles, the resolution of the gas grid should be so high that even Hal 9000 wouldn't be able to handle it or Sky-net would suddenly become self-unaware.

So I generated a simple smoke sim using H13 Smoke Solver. I didn't use Pyro cause it was probably overkill for such a simple sim. Then I wedged and cached to disk 10 versions of the smoke sim varying parameters like the turbulence amplitude and twirl radius (finding out what parameters to wedge is an art on its own).

Once I had 10 different smoke sims, I had 10 different velocity field sequences that I can use to advect my points.
Now, two ways to advect points in Houdini are:

  • create a POP system in a DOP Network and use the POP Advect by Volumes DOP node making sure to point it to the velocity fields cached on disk and re-imported somewhere else at the SOP level.
    PROs: POP land provides a large range of nodes to control particle motion.
    CONs: Slow
  • convert the velocity to VDB, merge the fields into one single VDB vector field (with VDB Vector Merge - don't worry if Houdini complains cause the grid components are different), then use VDB Advect Points in a SOP Solver to advect your points.
    PROs: Very fast.
    CONs: you've to do everything yourself.
    • PRO of this CONS: you CAN do whatever you want  :) !

When I mentioned thet the POP is slow, I still mean that it can process several hundreds of thousands of particles per second. The VDB solution can handle millions of points per second. Because of this, you can avoid rendering different versions of the particle sim and compositing them in order to smooth out the "particular" look (which is the main challenge of ink in the water effect).

So I decided to adopt the VDB solution.

In order to add extra points in the areas where the particles were more sparse, I added an Attribute Wrangle node in the end of the chain and I called it Fill Gaps. Since Attribute Wrangle works in the CVEX context, I am given the opportunity to add (or destroy) geometry.
The purpose of the script is basically adding points where there aren't.

This is the content of the script (make sure to create the proper UI parameters before sourcing the script) and feed the same point geometry both in input 1 and 2 of the Attrib Wrangle node.

Note: this fill gap algorithm is FAR from being perfect. Plus it just really fills the spaces between points larger than "mindist" within "seachrad". So the results might not be as expected. The good thing is that the added points remain surprisingly consistent during the animation, so , at least, you don't see crazy artifacts or points popping in suddenly, in the render.

float searchrad=ch("searchrad");
float mindist=ch("mindist");
int maxpoints=ch("maxpoints");
int fillpoints=ch("fillpts");

vector clpos;
int handle=pcopen(@OpInput2,"P",@P,searchrad,maxpoints+1);
int i=0;
    if (i==0) // the first point found should be the closest, in this case, itself. We want to skip it.
    if (length(@P-clpos)>mindist)
        vector pointstep=(clpos-@P)/(fillpoints*2+1); // this ensures there are no duplicate point
                                                                                  // at the cost of doubling the fill points number
        for (int t=0;t<fillpoints;t++)

After rendering 10 exr sequences, I imported them in Nuke and used them as sprites on a particle distribution scattered over a large plane, and rendered with DOF. Kinda slow actually, and the DOF in comp is always a PITA (has this improved in the last 15 years ? nope!).

This is the result.
Oh, there is a card disappearing in the last 10 frames, probably one of the Nuke particles ran out of fuel.

Ink in Water - test from Alessandro Pepe on Vimeo.

Thank you for reading !

Thursday, May 29, 2014

Houdini - Explicit Cache

Classical Scenario:
You deadline is today at 7pm. Which means your deadline really is tomorrow around 9pm ! You've time to recalculate your huge flip sim that takes about 13 hours to complete (note, not 12 hours, not 14...13 the evil number). Each bgeo file is about 666mb. You hope you'll have enough HD space, but hey ... HD space is never enough ! Even Confucius knew this.
So you cross your fingers, and hit "Render" on your rop_geometry network to start your 999 frames FLIP sim.

Now usually, when you've only one chance to do it right and if you miss it you'll be fired, usually this is what happens in order:
1 - you run out of HD space on the frame 998.
2 - Linux crashes for the first time in 12 years
3 - Houdini crashes for the 13th time in 2 hours
4 - power outage in the whole <city where you are in that moment>

You better remember that one of these 4 things (if not all of them) will happen. It's important to be positive.

How will our fellow FX Artist save his job, and manage to pay the rent of his mansion with pool in the Hollywood Hills ?

The answer is this gorgeous little gem of pure love called "explicit cache" and his little friend "explicit frames to cache" on the DOP Network node.

This option is off by default, cause the files generated can fill your 12Tb HD very quickly.
But this is where the second option comes really in handy !
You don't have to save ALL the .sim files for each one of your 999 frames of simulation.
You can save the last, say, 5 (or less maybe) and restart the sim from the smallest frame number in your cache ! This way you'll not clog your HD with unnecessary GIGANTIC .sim files, and you'll still be able to resume your sim.

n.4 happened. And I managed to simulate only 362 frames of my 999 frames sim. But I was wise enough to enable the explicit cache option and specify a frame history of 5.
So, if I go in my ...../simcache directory I'll see these 5 files:

What I usually do is delete the last one. Why ? Well ... I love Houdini, but I will never trust that he managed to write out the last sim file while the power of my computer was off. And you've a pretty good clue of this checking the file size of the files. They are all ~153 Mb, apart from the last one ! MM ... suspicious. Delete !

Perfect. Now you have 4 cache files and you're sure they are fully functional.

Now you can restart your sim starting from the frame 358 and Houdini will seamlessly continue simming like if you started from frame 1.

This saved my <censored> several times already.

Friday, July 19, 2013

(just another) Houdini FLIP water sim

Houdini fountain water sim - FLIP solver test from Alessandro Pepe on Vimeo.

A few days ago I realized the world couldn't survive without my contribution to the countless water simulation tests out there. Honestly this is not better or worse than many others but I had so much fun working on it.
I spent about 5 days overall between setup and render.

The workflow is pretty much the same explained in the SideFX Waterfall tutorials, with some minor changes in the water shader, and the creation of a wet map. Furthermore I paid extra attention to the bubbles underwater.

For the wet map I used Attribute Transfer SOP to transfer the "wetness" attribute from the particles generated by the Flip Simulation (after caching only the ones close to the fountain walls) to a dense particle object scattered on the fountain. The attribute transferring was performed in a DOPs Network (via SOP Solver), in order to preserve the previous state.

Initially I created 2 different wet maps:
  • WETNESS - This is a wet map that dries very quickly and reveals only the water spec on the fountain shader.
  • DARKNESS - This wet map is identical to the previous one for what concerns the shape, but dries way way slower and is in charge of keeping the fountain shader just darker.
Eventually I ended up using only the "DARKNESS" wetmap cause the light position didn't show any wet part of the fountain revealed by the water, (unfortunately) so ... useless ! But I thought it was a good idea to illustrate both wetness (in cyan) and darkness (in blue), in the pass below.

Houdini Wet map from Alessandro Pepe on Vimeo.

Monday, June 24, 2013

stuff growing and crawling - Venations - HDK version

I know, it's time to move on, enough with this growing and crawling stuff, but ... I just can't ! It's too much fun to see those tentacles growing and reaching out drawing crazy shapes in the virtual space. The problem with the previous version was that ... the algorithm is quite heavy because it relies upon two nested loops (for each seed, search for the closest root) : it was too slow.
So I decided it was about time to learn a bit of HDK and implement a prototype in C++.
This version is about 30 times faster and allows a real time feedback for way more complex structures than the OTL version.
In this video I am illustrating how to use this SOP node to create a simple tree and in the second part of the video , how to paint venations on a polygonal head.

Venation System - HDK version - demo from Alessandro Pepe on Vimeo.

The venation algorithm is based on this paper by Adam Runions.

Wednesday, June 12, 2013

HDK Learning Notes - quick start on Linux

(tested on Linux - CentOS 6.4 and Ubuntu 13.04)

Install compiler and libraries (as mentioned here)

$ sudo yum install tcsh gcc-c++ mesa-libGL-devel mesa-libGLU-devel Xi-devel
(if yum cannot find Xi-devel, try libXi-devel)
$sudo apt-get libgl-dev libglu-dev libxi-dev

To initialize houdini environment variables and commands:

$ cd /opt/hfs12.5.371/
$ source houdini_setup
The Houdini 12.5.371 environment has been initialized.

Now you'll be able to use the command houdini, houdinifx, hcustom, etc.
The command hcustom is a little wrapper to g++ that will take care of searching for houdini sdk headers and linking with the correct libraries:

$ hcustom SOP_mynode.C

... if we need OpenVDB (see below for installing OpenVDB headers)

$ hcustom -I /tmp/OpenVDB/include/

When you try to compile the first time you might get this error:

/usr/bin/ld: cannot find -lXi

which might be related to the fact yum wasn't able to install the library Xi-devel in the first step. In that happens, try this:

sudo yum install libXi-devel


If you plan to use OpenVDB you've to install the OpenVDB headers , cause they are not shipped with Houdini ( so far. 
Download the library and unzip it somewhere (for instance in /home/$USER/Download).

$ cd /home/alex/Download/openvdb
$ make clean
$ make install

... this will install the libraries into /tmp/OpenVDB

You might get the following errors:


io/ error: zlib.h: No such file or directory
  • edit the file /openvdb/io/
  •  change the line
    #include <zlib.h>
    #include </opt/hfs12.5.371/toolkit/include/zlib/zlib.h>

cmd/openvdb_view/Viewer.h:50:21: error: GL/glfw.h: No such file or directory
  • edit the file /openvdb/Makefile
  •  search for line starting with

    and remove vdb_view (it's on the same line of "install :")

/bin/bash: doxygen: command not found
  • $ sudo yum install doxygen

Monday, May 20, 2013

stuff growing and crawling - Hyphae - OTL version

I've been playing with path finding algorithms recently and I came up with an OTL that creates a venation structure given root points, seed points and an optional collision surface.
This can be used to create venations on, inside surfaces, and I guess some more cool stuff that at the moment I cannot even think of !

This is a quick demonstration video of the otl (I speeded it up 2x so you don't fall asleep while you watch)

A few days ago I was reading this article cause I was interested in reproducing Jack Frost effect from Rise of the Guardians.
This line of the article really caught my attention:
Our developer on that would take a model, whatever the frost needed to grow on, and he would run a cellular automata simulation across it
Cellular automata ? what the hell is that ? So I started researching a bit and as it always happens...the by-products of the research are usually even more interesting than the destination.

So I came across this video:

Check out this blog out when you have time, there is a lot of really interesting stuff, presented in a really nice way. 
Researching a bit more I found the details of the algorithm in this amazing paper:

(thank you Adam Runions, Martin Fuhrer, Brendan Lane, Pavol Federl, Anne−GaĆ«lle Rolland−Lagan, and Przemyslaw Prusinkiewicz for this amazing paper !!!)

This paper blew my mind. Bye Jack Frost (for now), welcome Hyphae ! 

So I quickly implemented a prototype in Houdini.

The Algorithm

Given this initial data:
  • hormone points (H)
  • roots points (R)
The target of the algorithm is to add points to R till the branches reach all the points in H. When one of the R branches reaches one of the H points, that H point will be removed by the group H and not used anymore for the next iteration.
So the way I implemented it in Houdini is the following:
  1. for each point h in H
    1. find the closest point cp in R and store (on h) its index (integer CLIDX) , location (vector CLPOS) and normalized vector pointing to it from h (vector CLDIR)
    2. if the cp is too close to h flag it for deletion (int IGNORE=1)
  2. delete all points in H where IGNORE==1
  3. in H create groups GR_* based on CLIDX (using partition sop)
  4. for each group gr in GR_*
    1. find the average CLDIR and normalize it --> AVGCLDIR
    2. create a new point r=CLPOS-AVGCLDIR
    3. merge the new r point into R
  5. goto 1
Test 1
"let's see if it works" 
H (hormones) = 100 points scattered by a sphere volume centered at the origin 
R (roots) = 1 point at the origin

 Test 2 
"hell yeah it does ! about more roots ??" 
H = same as before 
R = 3 points (manually placed randomly in the space not too far from the sphere I used to generate H)

Test 3 
"this is amazing. How about a lot more roots ?"
H = same as before 
R = points scattered from the surface of a sphere larger than the one I used to generate H

Uhmm...that is not so cool. Why ?

Problem 1: 
Well, to start we notice that not all of the points R actually grow. This is because of the step 1.1 in the algorithm ! Only one point between all R is chosen to be the closest one to a specific point in H. So some point of R might not be the closest to ANY H since the beginning.

1. we could decrease the number of R but that's cheating , isn't it.
2. we can increase the radius and number of the H distribution so that the outer H points are closer to the roots.

Problem 2: 
The veins grow radially towards the center, but as soon as they find a hormone, they stop without splitting any new branch. So we don't get that awesome intricate complexity we like so much and that doesn't let us sleep on night. The reason for this is point 2 in the algorithm. What happens is that we have so many veins now, that they take care of 'killing' the few H very quickly (and more or less at the same time, cause the distribution of the chasers (R) is similar to the distribution of the chased (H), since both are spherical distributions).

We need to add more H. The best option is to add more density the more we move towards the center of the spherical distribution of H. A quick way to do that is use Poly to VDB sop, check "fill interior" and scatter from this.

Problem 3: 
Lines are too straight ! We like chaos, variation !

How about adding in the algorithm some noise ?
Let's modify the step 4.B in the algorithm adding some noise fuction of the position.
Something like this:
4.2 --> r=CLPOS-AVGCLDIR+noise(CLPOS)

Test 4

H = 5000 points scattered from VDB volume with interior filled.
R = same as test 3
I've added a noise function as described in the solution of problem 3 above.

Collision Objects

This algorithm allows creating any sort of volumetric venation , which is really cool.
So I wanted to test it on a real complex model, and see if I could create the effect of venations growing and crawling over the surface of an object.

So I took the model of an arm from internet , I scattered points over the surface of the arm (H points), and I placed one single root point on the tip of the index finger. When I ran the simulation I noticed that, even if the result was mind blowing, there was a big problem : the venation was growing even "inside" the arm not only on the surface. I had to expect this of course, since the each loop iteration of the algorithm is performed on ALL the H points. So it doesn't really matter if the root is initially on the index finger tip : it will be affected even from the H points on the fore arm. So the venation would grow regardless.
You can see it even from another point of view : the algorithm is not aware about the surface of any model ! All we feed into him are point clouds.

So I decided to include in the algorithm the concept of Collision Objects: when a root point enters in a collition object (SDF sample value < 0) then it will be pushed out of the collision object along the normal of the collision object in that point (SDF gradient value).

So all we need is modify once more the step 4.2 of the algorithm:

4.2 --> r=CLPOS - AVGCLDIR + noise(CLPOS) + CVEC

CVEC = if ( sample (CLPOS,cobj) < 0 , -sample(CLPOS, cobj)*gradient(CLPOS,cobj) , 0 )
cobj is the collision object

Now CVEC component will make sure that the roots will not grow into the collision object.

What if we want to make sure the roots never leave the surface ?

For instance in this case, we don't want to see roots growing in the air. We want them to stick on the surface.
That's easy to fix : we just have to remove the condition to the CVEC component.
In other words, now the roots will be pushed outside of the collision object if they're inside, and towards the collision object if they're outside.

CVEC = -sample(CLPOS, cobj)*gradient(CLPOS,cobj)

This will make sure that the roots stick to the collision object.

In the following example, I duplicated the arm , poly-extruded it out so that the copy is slightly bigger than the arm. Then I selected the hand and a few more polygons on the arm and poly-extruded it again so that the selection is inside the original geometry. Then used it (the copy) as a collision object.
This way I created very quickly a small path on the arm to allow the root to reach the hand which is free to be filled with veins.
This is the collision object (in red) and the arm (in yellow):

This is a quick test

The Hyphae algorithm is great to create vines / venations structures using STATIC point clouds. What I mean is that, moving one single H or R point in the whole system can radically change the outcoming shape.

So, if you plan to have H, or R or both, animated and expect to have a smooth animation of the resulting venation system, this is not going to happen. The vines will be radically different between two adjacent frames. (this is great if you have to create some electricity effect, or lightnings, or voltaic arcs a la Frankenstein etc..).

The workaround to smoothly animate the resulting venation is to generate it on static point clouds, then deform it using some other deformer (Lattice for instance).

Sunday, May 19, 2013

stuff growing and crawling - L-Systems

In this nth experiment I wanted to create a way to wrap a growing geometry on the surface of another geometry without relying upon UVs on the target geometry.
The OTL I created takes 3 inputs:
  1. lsystem (this will be wrapped on top of 2)
  2. poly geometry
  3. one point

L-System (points and edges)

The L-System provided as first input can be any L-System where the root point of a new branch overlaps the point it generates from. In other words if you 'window' select the first point of a branch in your L-System you should end up selecting 2 points. Inside the OTL the L-System is pre-processed adding point attributes (like an attr that specifies if a certain point is a branch root, or the direction of each segment) that will make traversing it's hierarchy later on way faster. Something like this:

Wrapped Geometry (any geometry)

The OTL wraps the original L-System around this geometry. It can be any geometry. This geometry will be converted into a VDB SDF in the OTL, sample and gradient volume vops are used to quickly find the closest point on this surface. Because of this approach, the OTL doesn't require any UV coordinate to locate positions on this geometry and it's super fast since the SDF is calculate only once at the beginning of the whole simulation and contains already all the info to calculate the closest point on the surface with just a couple of multiplications and sums.

Start Point (one point)

The closest point of the Start Point on the Wrapped Geometry is where the root of the 'wrapped' L-System will sit. This point must be as close as possible to the the Wrapped Geometry surface (in order to have it within the Wrapped Geometry SDF VDB voxels).

How it works

simple otl interface
The original L-System is flattened on the XY plane before being processed. The root (the first point calculated) of the wrapped L-System (WL) is the closest point of the Start Point (SP) on the Wrapped Geometry (WG).The initial direction is given by the vector specified in the OTL UI (see image above).Every 'next' point position is calculated starting from the position of the previous point. For this reason I used a foreach loop making sure to uncheck the parameter "merge", since I want the same data to be processed and provided to the next cycle after adding a new point.To calculate the closest point on the surface from a certain point in the world space, I've used VDB SDF. SDF are already a blessing provided by Houdini but they've always had a certain level of imprecision. VDB SDF are way faster to calculate, and much more precise (thank you DreamWorks Animation for this gift !). The following is the VOP SOP structure to calculate the closest point on surface, given an SDF volume and a sample location in world space:


In this example I've created the wrapped L-System, and then used a Ray Sop to make sure each point really was on the surface, once per frame (Ray Sop is very fast).
I made sure to preserve the Width attribute calculated by default in the L-System Sop and I used it in the PolyWire Sop to drive the width of the Tubes as explained in the PolyWire documentation:
....This node works like the Wireframe node, but this node creates more complex tube geometry from curves, with smoother bends and intersections than Wireframe, especially for L-systems. The four numerical parameters support all the local variables of the Point operation, plus the LSYSTEM specific variables of $WIDTH, $SEGS, $DIV, $LAGE, $GEN, and $ARC....

WL has been calculated on a static WG (I choose the first frame of the hand animation). Then I used a Lattice Sop to wrap WL on the animated version of WG. Before applying the Lattice Sop I've calculated and stored the density of the points on a point attribute on the animated WG via VOP SOP where I used Point Cloud vex nodes to access the same geometry and return the number of points within a certain radius. Then I've used the density attribute to drive the size of the metaballs internally used by Lattice Sop.
Note: L-Systems are perfect to create intricate veins systems. Because L-Systems are a parallel rewriting systems, their size might get huge very fast. For instance in the previous example the original L-System had 95 generations and ~66k points. Nevertheless the generation of the L-System itself was very fast (less than 1s per frame for 95 generations).

OTL processing time to calculate the Wrapped L-System in this example:

Frames 1-84 (83 frames) : less than 2 minutes
Frames 85-91 (7 frames) : ~1 minute
Frames 92-96 (4 frames) : ~1 minute
Frames 97-99 (2 frames) : ~1 minute
Frames 100-102 (2 frames) : ~1 minute
Frames 103-104 (1 frame): ~1 minute
Frame 116 : ~2 minutes
Frame 122 : ~3 minutes
Frame 131 : ~5 minutes
Frame 138: ~10 minutes
Frame 145: ~21 minutes

Time to calculate 145 frames ~ 3h 40m.
on an Intel Core i7 3.0Ghz - 4 cores - 8 logical All VOP SOPs are set to 1 thread per proc.

As you can see , processing times increase exponentially when the number of L-System generation starts getting crazy so if the idea is, for instance, to cover a full animated character with veins, the best approach is divide and conquer, instead of using one single massive L-System.
Now, I guess this is just a way to accomplish this effect, I am sure there are many others I've not thought about. So, feel free to comment and pitch ideas !