Basic Embedding

[Richard Kain]

I haven't been posting recently, mainly because of other projects and time commitments. Life happens.

One thing that I've been doing recently though, is learning how to program in C++! Finally got over the learning hump that had been holding me back for so many years, and over the past few weeks I've been gobbling up basic C++ ravenously. Got everything from classes, extension, basic pointer and memory mapping, command-line build systems (cmake), arrays and their bizarre handling syntax, and some remedial OpenGL.

And as always, one of my favorite experiments when learning anything new is to embed Tilengine in it. It was actually even easier than usual, as Tilengine's C-nature makes it relatively painless to use it in conjunction with C++. (just include and link, and then you can call the C functions directly) For this one, I created a custom SDL window with an OpenGL context, and then mapped the output from Tilengine to a textured OpenGL quad. Here's the line of code that really makes the magic happen.

Code:

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, targetWidth, targetHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, pixelsPointer);

This is the OpenGL function that draws a pixel array to the currently bound OpenGL texture. Basically, you create a texture, create a pixel array, target Tilengine to the Pixel array, update your frame in Tilengine, bind your texture, call that line of code, and then draw your texture to a 3D element. (in this case a simple quad) That simple.

It took a little bit of experimenting to get the right settings for the draw function, but I had a head start on that thanks to my previous attempts. Now that I've got Tilengine mapped into OpenGL, I can run things like fragment shaders on Tilengine's output. That will open a few possibilities that didn't exist before.

[megamarc]

Hi Richard!

Congratulations for your new achievements and integrations. What you're doing in OpenGL with C++ can be done in plain C too, in fact both Tilengine and OpenGL are C libraries. The ability to apply GPU-based shaders at the output opens the possibility of implementing much better CRT/analog effects, the one I implemented does the job but it's quite basic compared to other alternatives.

Please let me know any help you may need

[Richard Kain]

It took a little doing, but I was able to get Tilengine running reasonably well with OpenGL GPU shaders. A big hurdle I ran across was dealing with the shaders themselves, and their restrictions regarding 8-bit graphics. As it turns out, a shader pipeline in OpenGL does NOT like 8-bit graphics, or anything formatted like 8-bit graphics. This proved to be a bit of a problem. For whatever reason, it treated the pixel-array output of Tilengine similarly, even though that output is not technically 8-bit. It would only accept graphics fully formatted as RGBA32, and was very particular about this.

Thankfully, I was already using SDL2 as my general framing library, and it has lots of tools for dealing with this sort of scenario. I created two SDL_Surface objects, formatted one of them as BGRA32, and pointed Tilengine at its pixel array for the actual pixel drawing. This turned out to work just fine, and be a viable container for storing the Tilengine-generated pixel data. I took the second SDL_Surface and formatted it as RGBA32. I then used a ConvertPixels function call to transfer the data from the Tilengine-rendered surface to the RGBA32 surface. Then I used the RGBA32 surface as the pixel source for updating the OpenGL Texture. This approach worked fine, and the OpenGL GPU shaders were able to use that data without issue.

The extra step was lamentable, and I would have preferred to copy the data over from Tilengine's output directly. The extra SDL_Surface tacks on a little extra used memory, and transferring the pixels over to a different format almost certainly eats into the performance. But the kinds of features that can be gained from utilizing GPU shaders are fairly significant, so I'm sticking with it for now.

[Richard Kain]

Whoooo, don't I feel like a fool. After typing up my previous entry, something occurred to me that I probably should have realized earlier. If you use the SDL_CreateRGBSurfaceWithFormat function in SDL2 to construct your "target" Surface for Tilengine rendering, and you set its format to SDL_PIXELFORMAT_BGRA32, you don't actually need the second surface. Your target surface may not be the exact same format as what the GPU shader wants, but it's close enough that conversion can be handled successfully by the glTexImage2D function. So having a second Surface is unnecessary, and can be safely skipped.

This is why it is useful to review your code. I went back and removed the second surface, and just do the pixel writing and converting with a single SDL_Surface that Tilengine targets directly. It's likely not quite as efficient as accessing a minimal array directly, but it's much closer than having a second extraneous surface lying around or trying to copy the contents of one array to another. Good stuff.

[megamarc]

Hi Richard!

As you say the surface for tilengine outbut must be 32-bit RGB + alpha, not 8 bit. The default windowing already uses SDL2 to create a valid texture:
https://github.com/megamarc/Tilengine/bl...dow.c#L227

Here you can se that the format is SDL_PIXELFORMAT_ARGB8888 . I did some experimentation with shaders a few years ago, I didn't dig deep but I remember they are quite flexible regarding data formats. I was experimenting with a possible implementation of the rendering core using shaders instead of software. But where the current software implementation runs anywhere, the shader-based one would require an OpenGL 3+ capable GPU with the EXT_gpu_shader4 extension for integer indexes and pixel data, that I don't know if it's even supported under GL ES 2.0 required for mobile and RPi. And speed gain wouldn't be dramatic either, as there would be lots of CPU -> GPU sync to properly implement de raster effects. So I eventually discarded it.

However Tilengine can be greatly enhanced with proper post-processing effect shaders, because the current CRT effect is quite basic and could be improved.

[Richard Kain]

I don't know if it's even supported under GL ES 2.0 required for mobile and RPi.

OpenGL ES 2's feature set is roughly analagous to OpenGL 3 for Desktop. As such, basic glsl shaders ARE supported by vanilla OpenGL ES 2. Also, the solution I came up with for using Tilengine to draw directly to an SDL2 surface seems to be working well, so the pixel data is not a concern. And I created a basic demo/proof-of-concept for running glsl shaders on the Raspberry Pi before I attempted to tie Tilengine into it. Not only can I run OpenGL shaders on the Pi, but I can also access and use the framebuffer for render-to-texture effects. I've also been building all of my trials and prototypes on the Raspberry Pi 2/3 during development. Everything I've made so far runs on the actual hardware.

Now, I don't necessarily think that the performance of this is going to be better than just using straight Tilengine. In fact, I kind of assumed that it wouldn't be faster. The point of this was to increase flexibility by adding desirable features. Keeping performance within manageable tolerances was my objective for dealing with frame-rates.

Having programmable shader support will allow for such things as color-keyed transparencies and potential integration with fore or back ground 3D rendering.

[Richard Kain]

I tried my current prototype out on my Rapberry Pi 3 last night. I had to make a few slight alterations to accommodate the shift to OpenGL ES 2. Mainly a few minor tweaks to the shaders. But yeah, I totally got it working, no real issues. Vertex Buffer Objects and glsl shaders, no problem. And Tilengine rendering to an SDL2 surface's pixel array. Good stuff.

[megamarc]

Good work!

Is there any public repository with your test, or are they it just private experiments? (I guess). I'm just staring with OpenGL ES in the RPi too, for a job project unrelated to Tilengine, but any shareable knowledge would be appreciated and put to good use. Of course it would benefit Tilengine too, as would move it towards more serious post-processing effects so important for a true retro-graphics experience.

[RootBeerKing]

Hey Richard Kain, any plans on doing an embed of Tileengine for Godot Engine? That would be fantastic!

[megamarc]

Hi RootBeerKing,

For this to happen, Godot itself must expose a way to load external libraries and expose them an ARGB surface where to draw. Does Godot engine have such feature? If so, can you please point us to the reference documentation explaining how to do it?

Thanks!