### Navigation

- index
- next |
- previous |
- NDK Dev Guide »
- 2D »

# Writing Image Readers & Writers

Readers and Writers are an intrinsic part of NUKE. These 2D operators allow us to overcome the problem of reading and writing to and from the ever growing existing file/image formats.

Knowing about the readers and writers architecture is therefore an important part of thirdparty NUKE development, which many times may involve custom file formats.

This section will go through the in’s and out’s of creating, understanding and using Readers and Writers.

## Architecture

Before we delve into the coding part of creating your own readers and writers it is important to understand how NUKE handles the reading and writing of files:

To identify which reader or writer version is needed, NUKE relies on parsing the file extension and finding a matching reader, so NUKE will always try to match up a file with a .mov extension to a movReader.<OS specific extension> reader.

There is still however a problem that can arise from a single format having multiple extensions, an example of this is JPG which can be both represented as .jpg and .jpeg, with just the previous system in place it would mean creating or duplicating the jpgReader extension to satisfy both jpg and jpeg. NUKE provides two systems to help you get around this problem one is through a TCL script file that it tries to invoke just before it attempts to find the reader in question and can be used to redirect a reader. This TCL script must have the same name as the source reader, in the case of .jpg to .jpeg the script would be called jpegReader.tcl , as for its contents it simply needs to redirect to the correct reader as following:

    # jpegReader.tcl
    load jpgReader

This script is invoked instead of jpegReader.<OS specific extension> and ensures the correctly named jpgReader.<OS specific extension> is loaded for the read.

The other system, is by extending the supported extension types for the reader itself ( will be explained below ).

## Readers

### The Reader Base Class

class `DD::Image::``Reader`

A Reader in the context of NUKE Thirdparty development provides a base class and interfaces to which NUKE can plug into in order to read a custom piece of data from disk. So in order to create a custom Reader the first step will be to create a new Reader which the base class is a DD::Image::Reader, there is a specialization of this class that can be used in its stead DD::Image::FileRead this class is a specialization of DD::Image::Reader and provides extend file management functionality, this will be explained in the next sub-section section link.

In order to create a custom reader using the DD::Image::Reader base class the derived class for the new reader needs to follow some specific steps and implement the basic interfaces.

The `DD::Image::Reader::Description` interface must be populated with the relevant information, this allows NUKE to identify and manage the readers and their supported extensions. This static member can be declared as follows:

    class MyReader : DD::Image::Reader
    {
      static const Description d;
      ...
    };

    static Reader* build( Read* r, int fd, const unsigned char* b, int n )
    {
        return new MyReader( r, fd, b, n );
    }

    static bool test( int fd, const unsigned char* block, int n )
    {
      return ( block[0] == MY_MAGIC_BYTE );
    }

    const Reader::Description MyReader::d( "myFile\0", "my file format", build, test );

In this example a class MyReader is derived from DD::Image::Reader, the derived class then declares a static Description member this description is what will control the construction and evaluate whether or not the MyReader is suitable to manipulate the current file, through the declaration of build and test in this case.

The previous implementation will look up for files with the extension .myFile, if this specific format a test is performed to ensure the format matches up to the extension which in this case will be to identify the MY_MAGIC_BYTE as the first element of the buffer. In the case of a misnamed format, NUKE will still test against the tester function. It falls then on the user to ensure the testing is sufficient to determine the file type is readable and avoid collisions with similar file formats.

As well as using the TCL solution suggested in the previous section, multiple extensions can be specified on a reader’s descriptions by constructing it with a NULL separated list for it’s first argument.:

    const Reader::Description MyReader::d( "myFile\0someOtherFile", "my file format", build, test );

In this example not only will the reader be used for files with a .myFile extension but also for .someOtherFile.

The constructor should use the `DD::Image::Reader::set_info()` to setup the area to be accessed from the file IO. This should ( if possible ) be done by a quick read operation. Complex reading, decoding and buffering can be done via `DD::Image::Reader::open()`.

The last function that needs to be overriden in order to create a basic working custom reader is `DD::Image::Reader::engine()`, this function will implement the actual reading of the file data and perform the user conversion into the NUKE native image format.:

    void MyReader::engine( int y, int x, int r, ChannelMask mask, Row& row )
    {
      row.range( 0, width() );

      for( int z = 0; z < 4; z++ ) {
        // A user implemented read function
        custom_read_channel( y, z, row.writable( z ) );
      }
    }

This code represents a basic `DD::Image::Reader::engine()` implementation, starts by setting the range of the row to the full width, and then reads all of the channels into the data row.

#### Other Interesting Methods

These are optiomal methods that may be implemented to provide extra functionality and information to your Reader.

Function | Description
void `DD::Image::Reader::``prefetchMetaData`() | This should be overriden to have frame specific metadata
const MetaData::Bundle& `DD::Image::Reader::``fetchMetaData`(const char* key) | A function that can be used to fetch metadata
bool `DD::Image::Reader::``supports_stereo`()`const` | Flag indicating whether it supports stereo or not
bool `DD::Image::Reader::``fileStereo`()`const` | Flag indicating whether this file is stereo or not
bool `DD::Image::Reader::``videosequence`()`const` | Flag indicating whether or not this file is a video sequence

### The FileReader Base Class

class `DD::Image::``FileReader`

FileReader is a class derived from Reader, it performs very much like its base class and a basic FileReader implementation must also implement the same basic interfaces ( see The Reader Base Class ). What makes DD::Image::FileReader interesting however is the added functionality it can provide, while creating your reader directly from DD::Image::Reader can entail having to also implement file manipulation functions ( this can ofcourse be desirable for specific systems ), the FileReader is a suitable solution for most implementation by providing functions to access and manipulate file data directly.

So in order to use this additional functionality we simply need to inherit from our custom reader from DD::Image::FileReader.:

    class MyReader : public DD::Image::FileReader
    {
      ...
    };

Now our reader can use the associated file handling helper functions associated with FileReader.

#### FileReader Extended Functionality Methods

The extended methods that are provided by the FileReader for extra functionality.

Function | Description
int `DD::Image::FileReader::``lock`(FILE_OFFSET offset, int min_length, int length) | Locks the data between the given parameters
int `DD::Image::FileReader::``lock`(FILE_OFFSET offset, int l) | Locks the data between the given parameters
int `DD::Image::FileReader::``lock`(FILE_OFFSET offset, unsigned int l) | Locks the data between the given parameters
`const unsigned char& DD::Image::FileReader::byte( FILE_OFFSET n ) const` | Accesses the byte at the given offset
`const unsigned char* DD::Image::FileReader::at( FILE_OFFSET n ) const` | Accesses the byte at the given offset
void `DD::Image::FileReader::``unlock`() | Unlocks a previous locked buffer
int `DD::Image::FileReader::``read`(void* p, FILE_OFFSET offset, int min_, int max_) | Reads directly from the file into memory
int `DD::Image::FileReader::``read`(void* p, FILE_OFFSET offset, int l) | Reads directly from the file into memory
int `DD::Image::FileReader::``read`(void* p, File_OFFSET offset, unsigned int l) | Reads directly from the file into memory

### Knobs on Readers

Adding Knobs to readers requires an extra step, because Readers do not behave like normal Ops there is a special object that can be initialized and registered with the Reader to give it the same capabilities as a regular op to create knobs. The two main steps to achieve this is to derive a custom object of type `DD::Image::ReaderFormat` the ReaderFormat is then capable of implementing the knobs() and knob_changed() interfaces, the second step is creating a new object of the right type within your custom reader and making sure the correct reader format is initialized.

The following code does just that

    class MyFileFormat : public DD::Image::ReaderFormat
    {
      // implementation of the knobs call
      void knobs(Knob_Callback c)
    }

    class MyReader : public DD::Image::FileReader
    {
      static const Description d;
    }

    static Reader* build( Read* r, int fd, const unsigned char* b, int n )
    {
      return new MyReader( r, fd, b, n );
    }

    static bool test( int fd, const unsigned char* block, int n )
    {
      return ( block[0] == MY_MAGIC_BYTE );
    }

    static ReaderFormat* buildformat(Read* iop)
    {
      return new MyFileFormat();
    }

    const Reader::Description MyReader::d( "myFile\0", "my file format", build, test, buildFormat );

The previous code now extends our reader to provide a ReaderFormat that can implement and extend the knobs collection for the MyReader object.

### Reading MetaData

Metadata is accessed using the same pattern as the decoding, the `DD::Image::fetchMetaData( const char* key )` can be implemented to return an object of type `DD::Image::MetaData::Bundle`.

    class MyReader : public DD::Image::FileReader
    {
      ...
      DD::Image::MetaData::Bundle& fetchMetaData( const char* key )
      {
        return _metaData;
      }

      DD::Image::MetaData::Bundle _metaData;
    }

In this example the _metaData object is returned when requested, this meta data is already pre-built somewhere else during object/reader construction.

### Testing with Tester

As it was previously mentioned ( see The Reader Base Class ) when a file is read NUKE tries to identify which reader to use based on the registered valid extensions for each individual reader, a second mechanism also exists to safeguard against misnamed files, the `DD::Image::Reader::(Tester*)( int fd, const unsigned char* block, int n )`:

    #define MY_MAGIC_BYTE 0x000000001
    static bool test( int fd, const unsigned char* block, int n )
    {
      return ( block[0] == MY_MAGIC_BYTE );
    }

The previous code is a simple test which will check the first byte of that and ensure it matches the defined MY_MAGIC_BYTE, this is ofcourse a very simple example of how to set it up, a truly usable implementation would check for a more complex bit pattern in order to ensure the format matches.

### Working Metadata

Metadata in NUKE is handled throught the `DD::Image::MetaData::Bundle` object, this abstract object provides all of the functionality needed to write and read data of integral types ( int, float, string... ).

There are two important things that need to be taken into account when handling MetaData, as these can be split into two categories, one is the NUKE recognized and defined metadata this is the case for time codes, where a specific key ( `DD::Image::MetaData::TIMECODE` ) should be used to store this value on a specific format ( you can learn more about keys here ). So when NUKE wants to read or write a time code this will be the key it will use.

The second type of metadata is the custom type, these will be recognized internally as simple key-value pairs and should be handled by on a case by case basis by the user.

For example if you decide to use a key named “somenamespace/timecode” instead of `DD::Image::MetaData::TIMECODE` to store your time codes, then you are responsible for maintaining and ensuring that data is accessed and written to the file where applicable.

Whenever possible it is always advisable to use the internal key descriptions so that the metadata can take advantage of any specific NUKE systems that use it.

## Writers

### The Writer Base Class

class `DD::Image::``Writer`

A writing a custom writer is an omologous part of writing a Reader, most of the time when you spend the time to write a custom Reader you will probably also want to have a Writer capable and bringing your new/modified work back onto your custom file format. To achieve this in the same way that we have developed a Reader ( see The Reader Base Class ) we can develop a writer by inheriting from the DD::Image::Writer , this ofcourse does not mean that you need to have a writer for your custom reader and vice-versa.

In order to have a basic Writer much like the Reader there are a set of interfaces that need to be implemented that provide the basic functionality and description of this new writer so that it can be managed by NUKE.

The first step to achieve this is to inherit the class from our `DD::Image::Writer` and make sure we implement a `DD::Image::Description` for the Writer:

    class MyWriter : public DD::Image::Writer
    {
      static const Description d;
      ...
    };

    static Writer* build()
    {
      return new MyWriter();
    }
    const Writer::Description MyWriter::d( "myFile\0", "my file type", build );

This code will setup a basic writer, we have created a new writer named MyWriter that derives from DD::Image::Writer within this writer we added a description, in our definition we can then declare and define a build function, this will be the factory method that will actually create the MyWriter object. The description is then populated with all of this relevant information, the writer will be triggered for files of extension .myFile unlike the reader there is no test function needed, since a writer should be able to write from NUKE’s internal format to any custom output data format.

In the same way as the Reader, we can extend the file extensions that will be associated with this writer by simply extending the file types ensure each element is separated by a NULL terminator. The writer does not suffer from misnaming problems since we can just choose another writer regardless of the extension we set for the file, the auto detect is simply for convenience.

In order to actually do any writing there are a few more intricate interfaces that we must implement, the main one being `DD::Image::Writer::execute()` this function should request the data from input0() into the chosen file, the file name can be retrieved using `DD::Image::Writer::filename()` and the frame using `DD::Image::Writer::frame()`:

    void MyWriter::execute()
    {
      // example implementation of execute
    }

This code will request the data from Input0() and attempt to write the data into the my file type format into filename(). Any errors can be reported on to `DD::Image::Writer::error()`.

#### Other Interesting Methods

This is a list of more advanced interfaces that you may want to implement for your custom writer and their description.

Function | Description
bool `DD::Image::Writer::``movie`()`const` | Flag indicating whether it is a movie or not
void `DD::Image::Writer::``finish`() | Should be overriden if the writer needs to perform any operations at finish
void `DD::Image::Writer::``knobs`(Knob_Callback cb) | Should be overriden to create writer knobs
int `DD::Image::Writer::``knob_changed`(Knob* knob) | Should be overriden to handle knob changed events
LUT* `DD::Image::Writer::``defaultLUT`()`const` | Specifies the default LUT for this writer
bool `DD::Image::Writer::``isDefaultLUTKnob`(Knob* knob)`const` | Flag indicating whether the LUT is default or not
int `DD::Image::Writer::``split_input`(int i)`const` | Flag indicating whether split inputs are present
const OutputContext& `DD::Image::Writer::``inputContext`(int n, OutputContext& c)`const` | Lets the writer override the output context

### The FileWriter Base Class

class `DD::Image::``FileWriter`

The FileWriter is a writer base class which extends the current DD::Image::Writer functionality by providing basic file handling functionality, in any case where the basic file handling functionality is needed this should be the class to derive for the implementation of the new writer as follows:

    class MyWriter : public DD::Image::FileWriter
    {
      static const Description d;
      ...
    };

By implementing the writer this way we are able to take advantage of the extended interface and file manipulation functions, unless there is a compeling reason or you are using some third party file manipulation, the FileWriter should be used as the base class.

#### FileWriter Extended Interface

Function | Description
bool `DD::Image::FileWriter::``open`() | Opens the output file
bool `DD::Image::FileWriter::``close`() | Closes the output file
bool `DD::Image::FileWriter::``write`(const void* p, FILE_OFFSET n) | Writes n bytes to the file.
bool `DD::Image::FileWriter::``write`(FILE_OFFSET n, const void* p, FILE_OFFSET o) | Writes n bytes given the offset.
bool `DD::Image::FileWriter::``seek`(FILE_OFFSET o) | Seek the file
FILE_OFFSET `DD::Image::FileWriter::``tell`()`const` | Returns where the pointer currently is on the file
std::string `DD::Image::FileWriter::``getTempFileName`(const char* pActualFileName=NULL )`const` | Returns the current temporary file name

### Writing Metadata

In order to access metadata to be written into your file, the Writer itself must evaluate the metadata from an input Iop, and call the corresponding fetchMetaData() , this data can then be used to modify or append to the file itself.

## Colourspace Handling & LUTs

In order to effectively create Readers and Writers for use with NUKE, it is also essential to have an understanding of how NUKE handles colourspace, this section will describe how NUKE handles colourspacing and LUTs in the context of Reads, Writes and during compositing.

### Viewing Data

While the intent of this section is not to explain the functionality associated with the viewer and compositing, it is essential to understanding NUKE’s handling of colourspace. NUKE assumes that any images to be displayed in the viewer are linearly colourspaced, this not only facilitates handling and calculations but also ensures a standard write format accross all supported file types, the viewer itself will apply a desired LUT to ensure that a preview is reliabily similar to the real output.

### Reading Data

As mentioned previously, NUKE expects all of its formats to be read into linear space, what that means is that the user is responsible for ensuring any colourspace conversions happen at the time of reading, once the reading is finished, NUKE will assume the colourspace is linear and will apply any operations under that assumption:

    void MyReader::engine( int y, int x, int r, ChannelMask mask, Row& row )
    {
      row.range( 0, width() );
      for( int z = 0; z < 4; z++ ) {
        // A user implemented read function
        custom_read_channel( y, z, row.writable( z ) );
      }
    }

In this example the custom_read_channel when writing the channel z would have to also perform the appropriate calculations to ensure the data is in linear colourspace.

### Writing Data

Writting data as an omologous operation to a read also has to make certain assumptions, all data is assumed to be in linear space when the reader is called, and any conversions need to be made on write, the difference here is that the colourspace output can be implicit or explicit, to achieve this there is a particular interface ( `DD::Image::Writer::defaultLUT()` ) that can be implemented for the Write that allows each write to specify a default ( implicit ) colour space. It is important to note that all other registered colourspaces are also available through the NUKE UI.

    LUT* MyWriter::defaultLUT() const
    {
      return LUT::getLut(LUT::FLOAT);
    }

This code will set the default LUT for the MyWriter writer to be FLOAT, as to when a file with a supported extension is detected and the default colourspace setting is selected this will be the output colourspace for the write.

### Navigation

- index
- next |
- previous |
- NDK Dev Guide »
- 2D »