Tuesday, July 24, 2012

idTech 4 (aka Doom 3) ES2.0

I've pushed my Doom 3 ES2.0 branch onto GitHub including the GLSL shaders. These shaders could also be used as a template to write new ARB programs for the minimum assets required by iodoom3. The shaders are licensed as GPLv3 and Doom 3 is licensed as GPLv3 + additional terms as defined by id Software.

Things are starting to liven up over on #iodoom3 (Freenode) and we'll hopefully start seeing some cool things  very soon. :-)

I've put a PayPal donate button on my blog; many people don't realize the time and effort that goes into these hobby projects. If this project, or other projects of mine have been useful to you, please do make a donation (even if it's a "gold coin donation" 1-2€ or 1-2$)

I'll stop shamelessly pimping myself out now and get ready for my day job.

Happy hacking!

27 comments:

  1. Is there a way to indicate which project one wants to support and would that make a difference?

    ReplyDelete
    Replies
    1. Absolutely, you can enter some comment during PayPal donation.

      Currently I'm focusing mostly on Doom 3, but you could for example indicate a particular feature or bug you'd like to see fixed sooner. I cannot provide you with any absolute guarantees (considering it's a donation.)

      Certainly if you donate and ask "please fix bug X" I'll try to prioritize that over bugs Y and Z. :-)

      If you want me to work on something other than my current project, Doom 3 ES2.0/GLSL, you could also note that in the comments, too.

      Delete
    2. On what kind of project are you able to work? mesa?

      Delete
    3. Generally my area of expertise is GNU/Linux and 3D graphics. Certainly Mesa; that's part of my day-job.

      Delete
  2. Have you heard of this?
    http://phoronix.com/forums/showthread.php?71665-Sponsor-An-Open-Source-Driver-Dev-For-a-1-a-Day

    Several Phoronix readers (me included) will be ready to pay for some work on Mesa.

    ReplyDelete
    Replies
    1. I had not heard about this initiative, unfortunately it probably falls too close to my day-job contract for me to be eligible. I'll keep an eye on it though.

      Delete
    2. It's just community-driven, nothing "official". But unfortunately I don't think the initiative will take form since it demand some works that probably nobody in the community is able to do.

      Nevertheless, if some independent dev provides some works and want to have a payback, some users will perhaps be happy to give via paypal.

      That said, I think most users are waiting for r600g performance improvements, this is perhaps not your area of expertise.

      Just for info, what's your job? (if you can talk about it).

      Delete
    3. I used to do some hacking on the Mesa r300 driver and reverse engineering tools (which really seems like a lifetime ago) so I am somewhat familiar with the Radeon architecture. However I've never developed for r600g.

      I work for Intel Finland Oy in the OTC group (Open Source Technology Center -- Intel has a fondness for 3-letter acronyms. :-P)

      Delete
    4. Ok so this probably not the best idea to work on AMD chipsets anyway ;-)

      Delete
  3. Hi Oliver!

    I tried emailing you earlier at "your name" @linux.intel.com but I'm not sure if that address is current or even ever correct so apologies if you get this message twice!

    I'm hoping to get your Maemo Q3 port working on Pandaboard but I can't get it to compile. To see the errors I'm getting (and hopefully respond too) please visit:

    http://phoronix.com/forums/showthread.php?72570-DarkPlaces-GLES-on-Pandaboard

    Which is a thread I've created to document my adventures in trying to get OGLES Quake working on the Panda.

    More OT with this blog posting, I've also tried compiling your idtech4 on the Pandaboard but didn't have much more luck than with Q3. You can see how far I got in the same Phoronix forum topic linked above.

    Thanks for your help!

    Dan

    ReplyDelete
    Replies
    1. Hi Dan,

      I checked my Maemo Quake3 sources, and discovered that I have introduced preprocessor define which should be enabled when compiling for ES1.0. You need to add `-DNOKIA' to your CFLAGS.

      Honestly I don't remember why I chose to use `#if !defined(NOKIA)' instead of one of the OpenGL ES1.0 preprocessor defines such as GL_VERSION_ES_CM_1_0.

      You might also want to check the Makefile for some optimization tips, at least adding -mfpu=neon may help things.

      Of course there are many places for improvement; I wrote a very very naive implementation for a tech demo, which somehow became something more...

      Delete
  4. Hi Dan,

    Yes, that address is correct however I typically only check my Intel addresses during the weekdays.

    Okay, first the easy question; you're compiling D3XP (Resurrection of Evil) which I typically do not test as much as I should; it's possible something is broken there. I'm running a compile now to confirm whether it's okay. I'll let you know the results in a bit.

    Oh, make sure you're using the `es2' branch, and not the `master' branch.

    I kind of vaguely remember fixing up some idHeap allocator templates, though those are included on the `master' branch. Really `es2' is the main development branch now and should be used as such; it's still possible to target desktop OpenGL on that branch (minus unimplemented GLSL features such as heat haze effects, for example.)

    Well, the build completed compiling both `d3xp' and `game'. It's really very hard to tell what's going on there without the complete build command and log, gcc version, etc. If you like you can email those to me and I'll try to check them for you tomorrow morning.

    I have some ideas about your Quake 3 build problems, but I need to go back and check the code on that; bit of a blast from the past. I'll post a new comment in a few moments.

    ReplyDelete
  5. Hi Oliver!

    Thanks for your quick response!

    I have now got your Q3 port to build on Pandaboard - it was actually very simple to do in the end - although I haven't got it running yet.

    If anyone else wants to compile Olivers Maemo Q3 port on their Pandaboard, all you have to do is change 'arm' on line 297 of the Makefile to 'armv7l' and on line 316 change -lGLES_CM to -lGLESv1_CM

    Maybe I haven't got this ioquake3 port running yet because I don't own Q3 so I'm trying to use it with rq3 instead like this:

    ./ioquake3.armv7l +set com_basegame Boomstick +set r_fullscreen 0

    Have you ever tried rq3 with your ioquake3 port Oliver? Could you try it if you've not done so already please?

    At first it was telling me it couldn't find default.cfg so I symbolically linked the baseq3 dir to my Boomstick base dir (which does run standalone with ioquake3 normally) but when I try that I get the following errors:

    ReplyDelete
  6. Hi again OM!

    I have to run ioquake with the '+set r_fullscreen 0' option as there seems to be a bug in the current TI Panda xorg driver (or OGLES lib?) that gives a black screen to anything trying to start full screen, or at least that is the case with dxx-rebirth.

    I was going to give you a link to the rq3.com download page too in case you don't have that mod already but rq3.com is down for me right now.

    Here's my error, which may not mean anything to anyone outside of the TI OMAP dev labs I do understand!

    ./ioquake3.armv7l +set com_basegame Boomstick +set r_fullscreen 0
    ioq3 1.36 linux-arm Jul 29 2012
    ----- FS_Startup -----
    Current search path:
    /home/dan/.q3a/baseq3/reaction1.pk3 (2961 files)
    /home/dan/.q3a/baseq3
    /home/user/baseq3

    ----------------------
    2961 files in pk3 files
    execing default.cfg
    com_zoneMegs will be changed upon restarting.
    execing q3config.cfg
    couldn't exec autoexec.cfg
    Hunk_Clear: reset the hunk ok
    ----- Client Initialization -----
    Couldn't read q3history.
    ----- Initializing Renderer ----
    -------------------------------
    QKEY found.
    ----- Client Initialization Complete -----
    ----- R_Init -----
    Initializing OpenGL subsystem
    PVR:(Warning): PVRSRVOpenDCDevice: Warning - 138 returned [95, /bridged_pvr_dc_glue.c]
    GLimp_HandleError: 0x3000: (null)
    Segmentation fault (core dumped)

    ReplyDelete
  7. Oh yeah! idtech4! :)

    I'll have another go at compiling it on my Pandaboard if you want but you need to give full instructions and you're correct in guessing that I didn't check out the es2 branch last time I tried as its not the default / whats linked on github. You need to make it clear either on here but even better on the github page how we should checkout and compile this idtech4 ES2.

    My guess for checking it would now be:

    git clone -b es2 https://github.com/omcfadde/idtech4.git

    Then build:

    cd idtech4/neo
    scons

    ???

    ReplyDelete
    Replies
    1. Yes, this should be okay. However you will need some minor patching for ARM, of course. Check out the `SConstruct' file. Good luck.

      Delete
  8. I get a similar error when I try with openarena instead of rq3:

    ./ioquake3.armv7l +set fs_basepath /usr/share/games/openarena +set fs_game baseoa +set r_fullscreen 0
    ioq3 1.36 linux-arm Jul 29 2012
    ----- FS_Startup -----
    Current search path:
    /home/dan/.q3a/baseoa
    /usr/share/games/openarena/baseoa/pak6-patch085.pk3 (559 files)
    /usr/share/games/openarena/baseoa/pak6-misc.pk3 (229 files)
    /usr/share/games/openarena/baseoa/pak5-TA.pk3 (139 files)
    /usr/share/games/openarena/baseoa/pak4-textures.pk3 (1753 files)
    /usr/share/games/openarena/baseoa/pak2-players.pk3 (669 files)
    /usr/share/games/openarena/baseoa/pak2-players-mature.pk3 (231 files)
    /usr/share/games/openarena/baseoa/pak1-maps.pk3 (100 files)
    /usr/share/games/openarena/baseoa/pak0.pk3 (1042 files)
    /usr/share/games/openarena/baseoa
    /home/dan/.q3a/baseq3
    /usr/share/games/openarena/baseq3

    ----------------------
    4722 files in pk3 files
    execing default.cfg
    couldn't exec q3config.cfg
    couldn't exec autoexec.cfg
    Hunk_Clear: reset the hunk ok
    ----- Client Initialization -----
    Couldn't read q3history.
    ----- Initializing Renderer ----
    -------------------------------
    QKEY found.
    ----- Client Initialization Complete -----
    ----- R_Init -----
    Initializing OpenGL subsystem
    PVR:(Warning): PVRSRVOpenDCDevice: Warning - 138 returned [95, /bridged_pvr_dc_glue.c]
    GLimp_HandleError: 0x3000: (null)
    Segmentation fault (core dumped)

    ReplyDelete
    Replies
    1. Replied on Phoronix. You shouldn't have any problems with using different media files (OpenArena, etc)

      PVR:(Warning): PVRSRVOpenDCDevice: Warning - 138 returned [95, /bridged_pvr_dc_glue.c]

      This means something is going wrong during OpenGL context initialization, far before any actual game code would be calling to the renderer.

      Delete
  9. Hi Oliver!

    Looks like you've deleted the post I made here yesterday with my Panda idtech4 build error? I'm not bothered either way really but it means I can't check if I'm getting the same error as I was before.

    Note this is building the ES2 branch but I've not modified SConstruct at all and nor am I actually concerned with running idtech4 on Panda but if you want to suggest some tweaks to SConstruct and we try again I'm up for that but Q3 is def. my main concern.

    g++ -o build/debug/d3xp/d3xp/ai/AI_pathing.os -c -fPIC -pipe -Wall -Wno-sign-compare -Wno-unknown-pragmas -fmessage-length=0 -fpermissive -fvisibility=hidden -g -O1 -D_DEBUG -fno-strict-aliasing -D_D3XP -DCTF -DGAME_DLL d3xp/ai/AI_pathing.cpp
    d3xp/ai/AI_pathing.cpp: In function ‘pathNode_t* BuildPathTree(const obstacle_t*, int, const idBounds&, const idVec2&, const idVec2&, obstaclePath_t&)’:
    d3xp/ai/AI_pathing.cpp:659:51: error: ‘next’ was not declared in this scope
    d3xp/ai/AI_pathing.cpp:659:55: error: a function call cannot appear in a constant-expression
    d3xp/ai/AI_pathing.cpp:659:57: error: template argument 2 is invalid
    d3xp/ai/AI_pathing.cpp:659:72: error: invalid type in declaration before ‘,’ token
    d3xp/ai/AI_pathing.cpp:667:16: error: request for member ‘Add’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:669:28: error: request for member ‘Get’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:669:116: error: request for member ‘Get’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:671:13: error: request for member ‘Add’ in ‘treeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:709:20: error: request for member ‘Add’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:713:20: error: request for member ‘Add’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:726:20: error: request for member ‘Add’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/AI_pathing.cpp:751:19: error: request for member ‘Add’ in ‘pathNodeQueue’, which is of non-class type ‘int’
    d3xp/ai/../../idlib/../d3xp/Game_local.h: At global scope:
    d3xp/ai/../../idlib/../d3xp/Game_local.h:116:11: warning: ‘NUM_RENDER_PORTAL_BITS’ defined but not used [-Wunused-variable]
    d3xp/ai/../../idlib/../d3xp/Game_local.h:810:11: warning: ‘CINEMATIC_SKIP_DELAY’ defined but not used [-Wunused-variable]
    d3xp/ai/../../idlib/../d3xp/Player.h:73:11: warning: ‘ASYNC_PLAYER_INV_AMMO_BITS’ defined but not used [-Wunused-variable]
    d3xp/ai/AI_pathing.cpp:1438:14: warning: ‘float HeightForTrajectory(const idVec3&, float, float)’ defined but not used [-Wunused-function]
    scons: *** [build/debug/d3xp/d3xp/ai/AI_pathing.os] Error 1
    scons: building terminated because of errors.

    ReplyDelete
    Replies
    1. Hi Dan,

      I didn't delete your comment. I honestly have no idea what happened there; I suppose it's possible it happened by accident, but I do not have any recollection of deleting the comment. I wouldn't do that.

      Could you please send me the output of `gcc -v' either as a comment here or on Phoronix?

      I just tried your command and it succeeds for me with gcc 4.6.2:


      oliver@omcfadde-MOBL1 ~/idtech4/neo (android) $ g++ -o build/debug/d3xp/d3xp/ai/AI_pathing.os -c -fPIC -pipe -Wall -Wno-sign-compare -Wno-unknown-pragmas -fmessage-length=0 -fpermissive -fvisibility=hidden -g -O1 -D_DEBUG -fno-strict-aliasing -D_D3XP -DCTF -DGAME_DLL d3xp/ai/AI_pathing.cpp
      d3xp/ai/../../idlib/../idlib/math/Vector.h: In function ‘void GetPointOutsideObstacles(const obstacle_t*, int, idVec2&, int*, int*)’:
      d3xp/ai/../../idlib/../idlib/math/Vector.h:281:32: warning: ‘bestPlane.idVec3::y’ may be used uninitialized in this function [-Wuninitialized]
      d3xp/ai/AI_pathing.cpp:163:16: note: ‘bestPlane.idVec3::y’ was declared here
      d3xp/ai/../../idlib/../idlib/math/Vector.h:281:32: warning: ‘bestPlane.idVec3::x’ may be used uninitialized in this function [-Wuninitialized]
      d3xp/ai/AI_pathing.cpp:163:16: note: ‘bestPlane.idVec3::x’ was declared here
      d3xp/ai/../../idlib/../idlib/math/Vector.h: In function ‘int GetObstacles(const idPhysics*, const idAAS*, const idEntity*, int, const idVec3&, const idVec3&, obstacle_t*, int, idBounds&)’:
      d3xp/ai/../../idlib/../idlib/math/Vector.h:270:28: warning: ‘nextEdgeNormal.idVec2::x’ may be used uninitialized in this function [-Wuninitialized]
      d3xp/ai/AI_pathing.cpp:351:57: note: ‘nextEdgeNormal.idVec2::x’ was declared here
      d3xp/ai/../../idlib/../idlib/math/Vector.h:270:28: warning: ‘nextEdgeNormal.idVec2::y’ may be used uninitialized in this function [-Wuninitialized]
      d3xp/ai/AI_pathing.cpp:351:57: note: ‘nextEdgeNormal.idVec2::y’ was declared here
      d3xp/ai/AI_pathing.cpp: At global scope:
      d3xp/ai/AI_pathing.cpp:1438:14: warning: ‘float HeightForTrajectory(const idVec3&, float, float)’ defined but not used [-Wunused-function]
      oliver@omcfadde-MOBL1 ~/idtech4/neo (android) $ echo $?
      0

      You may notice I'm on a different branch, however no changes have been made to the game or d3xp code. This error is very mysterious to me, but I will admit that I'm not an expert in C++ semantics yet.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. (repost has the <> in #include had disappeared)

      I had the same isseu, and found a solution on some other site (but sorry I didn't remember where exactly).

      You have to add #include "stddef.h"

      As the other AI_pathing.cpp has same issue, I updated the neo/idlib/precompiled.h file.

      I still don't understand where the issue comes from : difference between libc ?

      And I didn't take time to run the preprocessor only (with stddef.h includion and without) to fully understand the compilation error.

      Delete
  10. Hello Oliver,

    I cloned your branch and tried to run it on my Linux,
    I downloaded minimal assets pack (BTW do I need to patch your sources with this patch inside?)

    I installed Doom 3 and took pak000.pk4 from it (I need it, right?), then Im trying to run your code. (compiled simply by "scons" command) and I got:

    http://pastebin.com/wHwHKi1P

    Any ideas what could be wrong?

    ReplyDelete
    Replies
    1. It looks like you're missing some of the *.pk4 files, you should have the following copied from the Doom 3 CDs/DVD:

      total 1.5G
      -rw-r--r-- 1 oliver oliver 941K Jun 27 07:14 game00.pk4
      -rw-r--r-- 1 oliver oliver 1.5M Jun 27 07:21 game01.pk4
      -rw-r--r-- 1 oliver oliver 1.8M Jun 27 07:17 game02.pk4
      -rw-r--r-- 1 oliver oliver 944K Jun 27 07:16 game03.pk4
      -rw-r--r-- 1 oliver oliver 337M Jun 27 07:19 pak000.pk4
      -rw-r--r-- 1 oliver oliver 220M Jun 27 07:21 pak001.pk4
      -rw-r--r-- 1 oliver oliver 398M Jun 27 07:25 pak002.pk4
      -rw-r--r-- 1 oliver oliver 303M Jun 27 07:16 pak003.pk4
      -rw-r--r-- 1 oliver oliver 227M Jun 27 07:26 pak004.pk4
      -rw-r--r-- 1 oliver oliver 540K Jun 27 07:26 pak005.pk4
      -rw-r--r-- 1 oliver oliver 214K Jun 27 07:14 pak006.pk4
      -rw-r--r-- 1 oliver oliver 188K Jun 27 07:16 pak007.pk4

      The game pk4 files are not important since those will be replaced by the game pk4 files built by scons (you can use `scons BUILD_GAMEPAK=1' to make this process easier.)

      You must also clone the gl2progs repository, available on Github at git://github.com/omcfadde/gl2progs.git or https://github.com/omcfadde/gl2progs

      Everything (pak00*.pk4, gl2progs/, and game01-base.pk4 generated by scons) must be copied into a location accessible by Doom3: any directory listed in the "Current search path:" output of your log file.

      Doom3 ES2.0 will not run without gl2progs as these are a critical component of the GLSL backend. They are copyright and licensed as GPLv3+.

      Delete
    2. I am working on replacing the build system with GNU Autoconf/Automake which will make building and installation much easier. Until then, I'm afraid things are a little tricky.

      Delete
  11. This comment has been removed by a blog administrator.

    ReplyDelete
    Replies
    1. Quoting me verbatim, as your own comment, then linking to your own blog is rather bad netiquette. Don't do that.

      Delete