Google+ Followers

Monday, June 15, 2015

H14 - Point Wrangle vs Creep SOP

The following Vex code used in a Point Wrangle reproduces the behavior of the Creep SOP, and it's about 5 times faster.

I've used it to stick and move points on a NURBs surface.

Points To Stick --> Point Wrangle Input 1
Nurbs Surface --> Point Wrangle Input 2

vector newP=primuv(@OpInput2, "P", 0, set(v@P.x,v@P.y,0));
vector newN=primuv(@OpInput2, "N", 0, set(v@P.x,v@P.y,0));
The VEX command primuv is super fast.

This system works of course even with a polygonal object. Just make sure to create UV coords and N (using Facet Sop).

Wrangle nodes win again !

Thursday, July 17, 2014

Ink in Water

One way to achieve this effect is using a looot of particles advected by a velocity field generated by some gas simulation. Why not use the gas sim itself ? Cause in order to achieve the same detail offered by particles, the resolution of the gas grid should be so high that even Hal 9000 wouldn't be able to handle it or Sky-net would suddenly become self-unaware.

So I generated a simple smoke sim using H13 Smoke Solver. I didn't use Pyro cause it was probably overkill for such a simple sim. Then I wedged and cached to disk 10 versions of the smoke sim varying parameters like the turbulence amplitude and twirl radius (finding out what parameters to wedge is an art on its own).

Once I had 10 different smoke sims, I had 10 different velocity field sequences that I can use to advect my points.
Now, two ways to advect points in Houdini are:

  • create a POP system in a DOP Network and use the POP Advect by Volumes DOP node making sure to point it to the velocity fields cached on disk and re-imported somewhere else at the SOP level.
    PROs: POP land provides a large range of nodes to control particle motion.
    CONs: Slow
  • convert the velocity to VDB, merge the fields into one single VDB vector field (with VDB Vector Merge - don't worry if Houdini complains cause the grid components are different), then use VDB Advect Points in a SOP Solver to advect your points.
    PROs: Very fast.
    CONs: you've to do everything yourself.
    • PRO of this CONS: you CAN do whatever you want  :) !

When I mentioned thet the POP is slow, I still mean that it can process several hundreds of thousands of particles per second. The VDB solution can handle millions of points per second. Because of this, you can avoid rendering different versions of the particle sim and compositing them in order to smooth out the "particular" look (which is the main challenge of ink in the water effect).

So I decided to adopt the VDB solution.

In order to add extra points in the areas where the particles were more sparse, I added an Attribute Wrangle node in the end of the chain and I called it Fill Gaps. Since Attribute Wrangle works in the CVEX context, I am given the opportunity to add (or destroy) geometry.
The purpose of the script is basically adding points where there aren't.

This is the content of the script (make sure to create the proper UI parameters before sourcing the script) and feed the same point geometry both in input 1 and 2 of the Attrib Wrangle node.

Note: this fill gap algorithm is FAR from being perfect. Plus it just really fills the spaces between points larger than "mindist" within "seachrad". So the results might not be as expected. The good thing is that the added points remain surprisingly consistent during the animation, so , at least, you don't see crazy artifacts or points popping in suddenly, in the render.

float searchrad=ch("searchrad");
float mindist=ch("mindist");
int maxpoints=ch("maxpoints");
int fillpoints=ch("fillpts");

vector clpos;
int handle=pcopen(@OpInput2,"P",@P,searchrad,maxpoints+1);
int i=0;
    if (i==0) // the first point found should be the closest, in this case, itself. We want to skip it.
    if (length(@P-clpos)>mindist)
        vector pointstep=(clpos-@P)/(fillpoints*2+1); // this ensures there are no duplicate point
                                                                                  // at the cost of doubling the fill points number
        for (int t=0;t<fillpoints;t++)

After rendering 10 exr sequences, I imported them in Nuke and used them as sprites on a particle distribution scattered over a large plane, and rendered with DOF. Kinda slow actually, and the DOF in comp is always a PITA (has this improved in the last 15 years ? nope!).

This is the result.
Oh, there is a card disappearing in the last 10 frames, probably one of the Nuke particles ran out of fuel.

Ink in Water - test from Alessandro Pepe on Vimeo.

Thank you for reading !

Thursday, May 29, 2014

Houdini - Explicit Cache

Classical Scenario:
You deadline is today at 7pm. Which means your deadline really is tomorrow around 9pm ! You've time to recalculate your huge flip sim that takes about 13 hours to complete (note, not 12 hours, not 14...13 the evil number). Each bgeo file is about 666mb. You hope you'll have enough HD space, but hey ... HD space is never enough ! Even Confucius knew this.
So you cross your fingers, and hit "Render" on your rop_geometry network to start your 999 frames FLIP sim.

Now usually, when you've only one chance to do it right and if you miss it you'll be fired, usually this is what happens in order:
1 - you run out of HD space on the frame 998.
2 - Linux crashes for the first time in 12 years
3 - Houdini crashes for the 13th time in 2 hours
4 - power outage in the whole <city where you are in that moment>

You better remember that one of these 4 things (if not all of them) will happen. It's important to be positive.

How will our fellow FX Artist save his job, and manage to pay the rent of his mansion with pool in the Hollywood Hills ?

The answer is this gorgeous little gem of pure love called "explicit cache" and his little friend "explicit frames to cache" on the DOP Network node.

This option is off by default, cause the files generated can fill your 12Tb HD very quickly.
But this is where the second option comes really in handy !
You don't have to save ALL the .sim files for each one of your 999 frames of simulation.
You can save the last, say, 5 (or less maybe) and restart the sim from the smallest frame number in your cache ! This way you'll not clog your HD with unnecessary GIGANTIC .sim files, and you'll still be able to resume your sim.

n.4 happened. And I managed to simulate only 362 frames of my 999 frames sim. But I was wise enough to enable the explicit cache option and specify a frame history of 5.
So, if I go in my ...../simcache directory I'll see these 5 files:

What I usually do is delete the last one. Why ? Well ... I love Houdini, but I will never trust that he managed to write out the last sim file while the power of my computer was off. And you've a pretty good clue of this checking the file size of the files. They are all ~153 Mb, apart from the last one ! MM ... suspicious. Delete !

Perfect. Now you have 4 cache files and you're sure they are fully functional.

Now you can restart your sim starting from the frame 358 and Houdini will seamlessly continue simming like if you started from frame 1.

This saved my <censored> several times already.

Friday, July 19, 2013

(just another) Houdini FLIP water sim

Houdini fountain water sim - FLIP solver test from Alessandro Pepe on Vimeo.

A few days ago I realized the world couldn't survive without my contribution to the countless water simulation tests out there. Honestly this is not better or worse than many others but I had so much fun working on it.
I spent about 5 days overall between setup and render.

The workflow is pretty much the same explained in the SideFX Waterfall tutorials, with some minor changes in the water shader, and the creation of a wet map. Furthermore I paid extra attention to the bubbles underwater.

For the wet map I used Attribute Transfer SOP to transfer the "wetness" attribute from the particles generated by the Flip Simulation (after caching only the ones close to the fountain walls) to a dense particle object scattered on the fountain. The attribute transferring was performed in a DOPs Network (via SOP Solver), in order to preserve the previous state.

Initially I created 2 different wet maps:
  • WETNESS - This is a wet map that dries very quickly and reveals only the water spec on the fountain shader.
  • DARKNESS - This wet map is identical to the previous one for what concerns the shape, but dries way way slower and is in charge of keeping the fountain shader just darker.
Eventually I ended up using only the "DARKNESS" wetmap cause the light position didn't show any wet part of the fountain revealed by the water, (unfortunately) so ... useless ! But I thought it was a good idea to illustrate both wetness (in cyan) and darkness (in blue), in the pass below.

Houdini Wet map from Alessandro Pepe on Vimeo.

Monday, June 24, 2013

stuff growing and crawling - Venations - HDK version

I know, it's time to move on, enough with this growing and crawling stuff, but ... I just can't ! It's too much fun to see those tentacles growing and reaching out drawing crazy shapes in the virtual space. The problem with the previous version was that ... the algorithm is quite heavy because it relies upon two nested loops (for each seed, search for the closest root) : it was too slow.
So I decided it was about time to learn a bit of HDK and implement a prototype in C++.
This version is about 30 times faster and allows a real time feedback for way more complex structures than the OTL version.
In this video I am illustrating how to use this SOP node to create a simple tree and in the second part of the video , how to paint venations on a polygonal head.

Venation System - HDK version - demo from Alessandro Pepe on Vimeo.

The venation algorithm is based on this paper by Adam Runions.

Wednesday, June 12, 2013

HDK Learning Notes - quick start on Linux

(tested on Linux - CentOS 6.4 and Ubuntu 13.04)

Install compiler and libraries (as mentioned here)

$ sudo yum install tcsh gcc-c++ mesa-libGL-devel mesa-libGLU-devel Xi-devel
(if yum cannot find Xi-devel, try libXi-devel)
$sudo apt-get libgl-dev libglu-dev libxi-dev

To initialize houdini environment variables and commands:

$ cd /opt/hfs12.5.371/
$ source houdini_setup
The Houdini 12.5.371 environment has been initialized.

Now you'll be able to use the command houdini, houdinifx, hcustom, etc.
The command hcustom is a little wrapper to g++ that will take care of searching for houdini sdk headers and linking with the correct libraries:

$ hcustom SOP_mynode.C

... if we need OpenVDB (see below for installing OpenVDB headers)

$ hcustom -I /tmp/OpenVDB/include/

When you try to compile the first time you might get this error:

/usr/bin/ld: cannot find -lXi

which might be related to the fact yum wasn't able to install the library Xi-devel in the first step. In that happens, try this:

sudo yum install libXi-devel


If you plan to use OpenVDB you've to install the OpenVDB headers , cause they are not shipped with Houdini ( so far. 
Download the library and unzip it somewhere (for instance in /home/$USER/Download).

$ cd /home/alex/Download/openvdb
$ make clean
$ make install

... this will install the libraries into /tmp/OpenVDB

You might get the following errors:


io/ error: zlib.h: No such file or directory
  • edit the file /openvdb/io/
  •  change the line
    #include <zlib.h>
    #include </opt/hfs12.5.371/toolkit/include/zlib/zlib.h>

cmd/openvdb_view/Viewer.h:50:21: error: GL/glfw.h: No such file or directory
  • edit the file /openvdb/Makefile
  •  search for line starting with

    and remove vdb_view (it's on the same line of "install :")

/bin/bash: doxygen: command not found
  • $ sudo yum install doxygen

Monday, May 20, 2013

stuff growing and crawling - Hyphae - OTL version

I've been playing with path finding algorithms recently and I came up with an OTL that creates a venation structure given root points, seed points and an optional collision surface.
This can be used to create venations on, inside surfaces, and I guess some more cool stuff that at the moment I cannot even think of !

This is a quick demonstration video of the otl (I speeded it up 2x so you don't fall asleep while you watch)

A few days ago I was reading this article cause I was interested in reproducing Jack Frost effect from Rise of the Guardians.
This line of the article really caught my attention:
Our developer on that would take a model, whatever the frost needed to grow on, and he would run a cellular automata simulation across it
Cellular automata ? what the hell is that ? So I started researching a bit and as it always happens...the by-products of the research are usually even more interesting than the destination.

So I came across this video:

Check out this blog out when you have time, there is a lot of really interesting stuff, presented in a really nice way. 
Researching a bit more I found the details of the algorithm in this amazing paper:

(thank you Adam Runions, Martin Fuhrer, Brendan Lane, Pavol Federl, Anne−GaĆ«lle Rolland−Lagan, and Przemyslaw Prusinkiewicz for this amazing paper !!!)

This paper blew my mind. Bye Jack Frost (for now), welcome Hyphae ! 

So I quickly implemented a prototype in Houdini.

The Algorithm

Given this initial data:
  • hormone points (H)
  • roots points (R)
The target of the algorithm is to add points to R till the branches reach all the points in H. When one of the R branches reaches one of the H points, that H point will be removed by the group H and not used anymore for the next iteration.
So the way I implemented it in Houdini is the following:
  1. for each point h in H
    1. find the closest point cp in R and store (on h) its index (integer CLIDX) , location (vector CLPOS) and normalized vector pointing to it from h (vector CLDIR)
    2. if the cp is too close to h flag it for deletion (int IGNORE=1)
  2. delete all points in H where IGNORE==1
  3. in H create groups GR_* based on CLIDX (using partition sop)
  4. for each group gr in GR_*
    1. find the average CLDIR and normalize it --> AVGCLDIR
    2. create a new point r=CLPOS-AVGCLDIR
    3. merge the new r point into R
  5. goto 1
Test 1
"let's see if it works" 
H (hormones) = 100 points scattered by a sphere volume centered at the origin 
R (roots) = 1 point at the origin

 Test 2 
"hell yeah it does ! about more roots ??" 
H = same as before 
R = 3 points (manually placed randomly in the space not too far from the sphere I used to generate H)

Test 3 
"this is amazing. How about a lot more roots ?"
H = same as before 
R = points scattered from the surface of a sphere larger than the one I used to generate H

Uhmm...that is not so cool. Why ?

Problem 1: 
Well, to start we notice that not all of the points R actually grow. This is because of the step 1.1 in the algorithm ! Only one point between all R is chosen to be the closest one to a specific point in H. So some point of R might not be the closest to ANY H since the beginning.

1. we could decrease the number of R but that's cheating , isn't it.
2. we can increase the radius and number of the H distribution so that the outer H points are closer to the roots.

Problem 2: 
The veins grow radially towards the center, but as soon as they find a hormone, they stop without splitting any new branch. So we don't get that awesome intricate complexity we like so much and that doesn't let us sleep on night. The reason for this is point 2 in the algorithm. What happens is that we have so many veins now, that they take care of 'killing' the few H very quickly (and more or less at the same time, cause the distribution of the chasers (R) is similar to the distribution of the chased (H), since both are spherical distributions).

We need to add more H. The best option is to add more density the more we move towards the center of the spherical distribution of H. A quick way to do that is use Poly to VDB sop, check "fill interior" and scatter from this.

Problem 3: 
Lines are too straight ! We like chaos, variation !

How about adding in the algorithm some noise ?
Let's modify the step 4.B in the algorithm adding some noise fuction of the position.
Something like this:
4.2 --> r=CLPOS-AVGCLDIR+noise(CLPOS)

Test 4

H = 5000 points scattered from VDB volume with interior filled.
R = same as test 3
I've added a noise function as described in the solution of problem 3 above.

Collision Objects

This algorithm allows creating any sort of volumetric venation , which is really cool.
So I wanted to test it on a real complex model, and see if I could create the effect of venations growing and crawling over the surface of an object.

So I took the model of an arm from internet , I scattered points over the surface of the arm (H points), and I placed one single root point on the tip of the index finger. When I ran the simulation I noticed that, even if the result was mind blowing, there was a big problem : the venation was growing even "inside" the arm not only on the surface. I had to expect this of course, since the each loop iteration of the algorithm is performed on ALL the H points. So it doesn't really matter if the root is initially on the index finger tip : it will be affected even from the H points on the fore arm. So the venation would grow regardless.
You can see it even from another point of view : the algorithm is not aware about the surface of any model ! All we feed into him are point clouds.

So I decided to include in the algorithm the concept of Collision Objects: when a root point enters in a collition object (SDF sample value < 0) then it will be pushed out of the collision object along the normal of the collision object in that point (SDF gradient value).

So all we need is modify once more the step 4.2 of the algorithm:

4.2 --> r=CLPOS - AVGCLDIR + noise(CLPOS) + CVEC

CVEC = if ( sample (CLPOS,cobj) < 0 , -sample(CLPOS, cobj)*gradient(CLPOS,cobj) , 0 )
cobj is the collision object

Now CVEC component will make sure that the roots will not grow into the collision object.

What if we want to make sure the roots never leave the surface ?

For instance in this case, we don't want to see roots growing in the air. We want them to stick on the surface.
That's easy to fix : we just have to remove the condition to the CVEC component.
In other words, now the roots will be pushed outside of the collision object if they're inside, and towards the collision object if they're outside.

CVEC = -sample(CLPOS, cobj)*gradient(CLPOS,cobj)

This will make sure that the roots stick to the collision object.

In the following example, I duplicated the arm , poly-extruded it out so that the copy is slightly bigger than the arm. Then I selected the hand and a few more polygons on the arm and poly-extruded it again so that the selection is inside the original geometry. Then used it (the copy) as a collision object.
This way I created very quickly a small path on the arm to allow the root to reach the hand which is free to be filled with veins.
This is the collision object (in red) and the arm (in yellow):

This is a quick test

The Hyphae algorithm is great to create vines / venations structures using STATIC point clouds. What I mean is that, moving one single H or R point in the whole system can radically change the outcoming shape.

So, if you plan to have H, or R or both, animated and expect to have a smooth animation of the resulting venation system, this is not going to happen. The vines will be radically different between two adjacent frames. (this is great if you have to create some electricity effect, or lightnings, or voltaic arcs a la Frankenstein etc..).

The workaround to smoothly animate the resulting venation is to generate it on static point clouds, then deform it using some other deformer (Lattice for instance).