Image Processing

Blocks & Textures

ILIAD provides its own classes to manage pixel blocks and textures, those are located in the OdysseyImaging Module.

FOdysseyBlock

Note

FOdysseyBlock will certainly be removed in a near future. Please don’t use this class unless you really need to.

FOdysseyBlock is a simple class which makes the bridge between ILIAD’s imaging library (ULIS) and Unreal Engine. Its purpose is to be able to retrieve all pixels in a single Unreal Engine TArray<uint8>. Most of the time prefer using ULIS::FBlock instead of FOdysseyBlock.

FOdysseySurface

FOdysseySurface is the bridge between a pixel block (ULIS::FBlock) and an Unreal Engine 2D Texture (UTexture2D). It creates a link between them so that they can be synchronized.

The pixel block is easily editable and stored on the CPU and the UTexture2D is the bridge between CPU and GPU but is harder to edit (see Textures)

When the pixel block is dirty, the linked texture is refreshed with the content of the block. It manages all additional UTexture2D options, like : Hue Rotation, Luminosity, Color Adjustments, etc… Thos options are not applied on the pixel block.

ULIS

All Image Processing in ILIAD is powered by the ULIS Library, a library developped by praxinos to provide everything you need to edit any type of images with high performances.

For more informations on how ULIS itself is working, and what you can do with it, check ULIS documentation.

In ILIAD, ULIS imported through its own module : the ULIS Module.

That module purpose is just to give ILIAD access to ULIS header files and libs.

Usage

Access ULIS in a Module

Using ULIS in an ILIAD module requires access to the ULIS Module.

ILIAD also provides an ULISLoader Module.

ULISLoader provides access to ULIS contexts depending on the Image Format requested.

Using ULISLoader is not mandatory but highly recommended for most usecases, as it avoids you create a Context / ThreadPool / FontEngine / CommandQueue each time you need to use ULIS.

To grant access to ULIS and ULISLoader to your module, add their names in its Build.cs file :

Build.cs

PublicDependencyModuleNames.AddRange(
    new string[] {
        "ULIS",
        "ULISLoader"
    }
);

Most of ULIS Commands can be accessed through a ::ULIS::FContext object. ULISLoader provides us an easy way to retrieve a ::ULIS::FContext object according to the image format you need to work on :

Load ::ULIS::FContext

#include "ULISLoaderModule.h"
#include 

//Example with RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

Once you retrieve a context, you don’t have to delete it, ULISLoader takes care of that.

Note

ULISLoader can create several ::ULIS::FContext, but there will only be one existing context per Image Format.

Also, ULISLoader will create a single ThreadPool, CommandQueue and FontEngine which will all be shared among all ::ULIS::FContext objects.

ULIS Asynchronicity

Asynchrone Behaviour

Danger

All ::ULIS::FContext generated by ULISLoader has an Asynchrone Command Queue. That means each command is executed asynchronously from each other.

Example of the invalid ::ULIS::FContext because of its Asynchrone CommandQueue :

Invalid Use of Asynchrone

#include "ULISLoaderModule.h"
#include 

//--- The following lines are synchrone, no problem

//Retrieve the FContext for RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

//Create a block to work on
::ULIS::FBlock block(128, 128, ::ULIS::Format_RGBA8);

//Create colors to fill the block with
::ULIS::FColor red =  ::ULIS::FColor::FromRGBA8(255, 0, 0, 255);
::ULIS::FColor green =  ::ULIS::FColor::FromRGBA8(0, 255, 0, 255);
::ULIS::FColor blue = ::ULIS::FColor::FromRGBA8(0, 0, 255, 255);

//--- The following lines are asynchrone
// The Fill red command can be executed at the same time as the Fill Green and Fill Blue Command
// This can lead to crashes, but also can lead to invalid result without crashes
// In this case, the result will certainly be that the block pixels will each one randomly become red, green or blue.
ctx.Fill( block, red );
ctx.Fill( block, green );
ctx.Fill( block, blue );

//Moreover, you never know when a command finished
//Using block immediately as an input to another command, or just reading it
//will produce invalid results

To fix the above code, there is several solutions : - Using ::ULIS::FContext::Finish() method (lazy solution) - Using ::ULIS::FEvent (optimized solution)

Using ::ULIS::FContext::Finish()

::ULIS::FContext::Finish() is a method that allows you to wait until all previous commands are finished.

Here is how to fix the Asynchrone Behaviour code using ::ULIS::FContext::Finish() :

Fix Asynchrone using ::ULIS::FContext::Finish()

#include "ULISLoaderModule.h"
#include 

//--- The following lines are synchrone, no problem

//Retrieve the FContext for RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

//Create a block to work on
::ULIS::FBlock block(128, 128, ::ULIS::Format_RGBA8);

//Create colors to fill the block with
::ULIS::FColor red =  ::ULIS::FColor::FromRGBA8(255, 0, 0, 255);
::ULIS::FColor green =  ::ULIS::FColor::FromRGBA8(0, 255, 0, 255);
::ULIS::FColor blue = ::ULIS::FColor::FromRGBA8(0, 0, 255, 255);

//--- The following lines are asynchrone, but ctx.Finish() waits after each command so the next command relies on valid data

ctx.Fill( block, red );
ctx.Finish(); //Wait for the Fill with red command to finish

ctx.Fill( block, green );
ctx.Finish(); //Wait for the Fill with green command to finish

ctx.Fill( block, blue );
ctx.Finish(); //Wait for the Fill with blue command to finish

//Using block immediately as an input to another command, or just reading it is now valid as you know all the commands are finished

That solution works, but as you always wait for ALL commands to be finished, it will be non optmized for situations where you don’t need for absolutely all commands to be finished. (See Optimized Usage) The best solution to use is always using ::ULIS::FEvent.

Using ::ULIS::FEvent

::ULIS::FEvent allows you know when an single event is finished and can be provided to other commands to create a dependencies between commands.

Here is how to fix the Asynchrone Behaviour code using ::ULIS::FEvent :

Fix Asynchrone using ::ULIS::FEvent

#include "ULISLoaderModule.h"
#include 

//--- The following lines are synchrone, no problem

//Retrieve the FContext for RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

//Create a block to work on
::ULIS::FBlock block(128, 128, ::ULIS::Format_RGBA8);

//Create colors to fill the block with
::ULIS::FColor red =  ::ULIS::FColor::FromRGBA8(255, 0, 0, 255);
::ULIS::FColor green =  ::ULIS::FColor::FromRGBA8(0, 255, 0, 255);
::ULIS::FColor blue = ::ULIS::FColor::FromRGBA8(0, 0, 255, 255);

//Create the ::ULIS::FEvent objects you need
::ULIS::FEvent eventFillRed;
::ULIS::FEvent eventFillBlue;

//--- The following lines are asynchrone
// As FEvent defines dependencies, it will force every command to be executed only when its dependencies are finished
// Each call to ctx.Fill() are still asynchronous, it means ctx.Fill does not wait until its dependency is finished, the underlying commands does.
// That's why, even if you define dependencies, once every calls of ctx.Fill() are done, you still need to call ctx.Finish() to wait until every command finished.

ctx.Fill(   block,
            red,
            ::ULIS::FRectI::Auto, //a Rect parameter needed by the Fill command
            ::ULIS::FSchedulePolicy::CacheEfficient, //a Schedule Policy parameter needed by the Fill command
            0, //that command has to wait on 0 commands
            nullptr, //no command array to wait on
            &eventFill ); //retrieve the FEvent of the Fill Red command

ctx.Fill(   block,
            green,
            ::ULIS::FRectI::Auto, //a Rect parameter needed by the Fill command
            ::ULIS::FSchedulePolicy::CacheEfficient, //a Schedule Policy parameter needed by the Fill command
            1, //that command has to wait on 1 commands (the Fill Red Command)
            &eventFillRed, //a pointer on the list of commands to wait on (here there is just 1 command to wait on, just pass a pointer to this command directly
            &eventFillGreen ); //retrieve the FEvent of the Fill Green command

ctx.Fill(   block,
            green,
            ::ULIS::FRectI::Auto, //a Rect parameter needed by the Fill command
            ::ULIS::FSchedulePolicy::CacheEfficient, //a Schedule Policy parameter needed by the Fill command
            1, //that command has to wait on 1 commands (the Fill Green Command)
            &eventFillGreen, //a pointer on the list of commands to wait on (here there is just 1 command to wait on, just pass a pointer to this command directly
            nullptr ); //no need to retrieve here as there is no more commands afterwards

//Wait for the Fill with blue command to finish
//You could go on and on with FEvents but as there is no more commands to be done, juste call ctx.Finish() to wait for everything to be done.
ctx.Finish();

//Using block immediately as an input to another command, or just reading it is now valid as you know all the commands are finished

Optimized Usage

There is several way to optimize the usage of ULIS in ILIAD depending on the situation you are facing.

Optimized Asynchronicity

You can take advantage of the asynchronous behaviour of ::ULIS::FContext in some situations like : - Executing actions on several block which does not depend on each other - Executing an action on multiple non overlapping rects of a block

Examples :

Several blocks with no dependencies

#include "ULISLoaderModule.h"
#include 

//--- The following lines are synchrone, no problem

//Retrieve the FContext for RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

//Create some blocks to work on
TArray<::ULIS::FBlock> blocks;
for (int i = 0; i < 10; i++)
{
    blocks.Emplace(128, 128, ::ULIS::Format_RGBA8);
}

//Once the blocks are created, fill them with their own color asynchronously
//Their will be no conflict between each command as they're not working or depending on each other blocks
for (int i = 0; i < 10; i++)
{
    ::ULIS::FColor color =  ::ULIS::FColor::FromRGBA8(i * 20, 0, 0, 255);
    ctx.Fill( blocks[i], color );
}

//Wait for all Fills to be done
ctx.Finish();

Several rects in a single block

#include "ULISLoaderModule.h"
#include 

//--- The following lines are synchrone, no problem

//Retrieve the FContext for RGBA8
::ULIS::FContext& ctx = IULISLoaderModule::StaticFindOrAddContext( ::ULIS::Format_RGBA8 );

//Create a block to work on
::ULIS::FBlock block(1000, 128, ::ULIS::Format_RGBA8);

//Create some non overlapping rects to fill
TArray<::ULIS::FRectI> rects;
for (int i = 0; i < 10; i++)
{
    rects.Emplace(i * 100, 0, 100, 128);
}

//Once the rects are created, fill them with their own color asynchronously
//Their will be no conflict between each command as they're not working on the same area of the block
for (int i = 0; i < 10; i++)
{
    ::ULIS::FColor color =  ::ULIS::FColor::FromRGBA8(i * 20, 0, 0, 255);
    ctx.Fill( blocks[i], color, rects[i] );
}

//Wait for all Fills to be done
ctx.Finish();