Tracking Modifications

Tracking Modifications Between Sessions

When you are interactively working with a palette, Slim keeps track of changes that will require a regeneration and recompilation of the shader (or master) in order to take effect. Every time you render your scene, Slim checks which appearances have been modified since their shaders were last generated and regenerates/recompiles what is necessary.

For interactive work, this is a relatively simple process: every attachable appearance contains a flag indicating whether it is "dirty" or "clean." When an appearance is modified in a manner which will require changes to the shader (e.g. the value of an internal parameter is changed), it is marked as dirty. When the shader is regenerated, it is marked as clean.

Tracking this state becomes more complicated between sessions of Slim. If, when loading a palette, Slim were to assume that any existing shaders were invalid, you would always have to wait for those shaders to be regenerated. On the other hand, if Slim were to assume that any existing shaders were valid, you might unknowingly find yourself using old or invalid versions of the shader.

To maintain a consistent dirty state between sessions, Slim not only tracks that an appearance is modified, it keeps track of when an appearance is modified. This modification time (measured in POSIX fashion) is stored within the body of the shader as a comment:

    /* shader modification timestamp: 1109976261 */

This same number is stored with the appearance when it is saved:

    function shadingmodel_aov "Glass" "pixar,Glass#1" {
        ...
        modified 1109976261
        ...
    }

When this appearance is next loaded, Slim compares the modified value in the palette to the timestamp stored within the shader:

  • If the numbers are equal, Slim knows that the last modification to the appearance as stored in the palette has been propagated to the shader on disk. In this case, the shader is determined to be clean and is not regenerated for the next render.
  • If the modified value stored in the palette is greater than the value recorded in the shader, there are changes that have been made to the appearance that have not been propagated to the shader. In this case, the shader is determined to be dirty and is regenerated for the next render.
  • If the modified value stored in the palette is less than the value recorded in the shader, there are changes that have been propagated to the shader that were not stored in the palette. This may have occurred if a user made changes to an appearance, generated the shader, but abandoned those changes by not saving the palette. In this case, the shader is determined to be dirty and is regenerated for the next render.

The modified mechanism is new for Slim 6.5. Slim 6.0 employed a similar mechanism known as mastermtime. This mechanism differed from the one described above in that, rather than storing the time that an appearance was modified, it stored the time that the shader was generated (more precisely, the mtime of the file). The mastermtime mechanism had two main problems:

  • Slim recorded the modification time of the file on disk for later comparisons. Unfortunately, because the mtime is updated when a file is copied, Slim would consider dirty any shader that had been copied to a new location for the sake of rendering. This resulted in shaders often being unnecessarily regenerated for rendering.
  • This mechanism did not work well if multiple jobs were simultaneously working with the shaders in a common directory. Because Slim was tracking and checking when the shader was generated, the shader generated by one job was deemed to be dirty by the next job to come along. This meant that jobs would often "stomp on" one another.

These problems have been addressed by the modified mechanism:

  • By recording the time within the body of the shader itself, the shader can be moved or copied and the time will not be affected.
  • By recording the time that the appearance is modified (rather than the time that the shader is created), jobs running simultaneously will all agree on the modification time, so long as they don't themselves perform any modifications to the appearance.

To understand the benefits of the modified mechanism, you may wish to perform this experiment:

  1. Check that your shader directory is clean of existing shaders
  2. Open the Slim Message Log, set the Filter to Info, and make sure the Timestamp box is checked.
  3. Create a new palette consisting of three Constant shaders and name them A, B, C.
  4. Change A's Surface カラー to red and perform a preview render.
  5. Change B's Surface カラー to yellow and perform a preview render.
  6. Change B's Surface カラー to green, but do not perform a preview render.
  7. Save the palette.
  8. Change C's Surface カラー to purple and perform a preview render.
  9. Close the palette without saving it.

Now reopen the palette and render icons for the three shaders. A, B, and C should be represented as red, green, and blue respectively, as seen here:

Now open the Message Log to see which shaders were regenerated/recompiled.
  • Because all modifications saved for A (the change of Surface カラー to red) were represented in its shader, A should not have been regenerated.
  • Because B had a modification that was saved in the palette (the change of Surface カラー from yellow to green) that was not represented in the shader, B should have been regenerated.
  • Because C had a modification that was represented in the shader (the change of Surface カラー from blue to purple) but not in the palette, C should have been regenerated.

Now close the palette without saving, reopen it, and render all of the icons. You should find that, because all of the shaders are now in sync with what had been stored in the palette, none of them were regenerated.