Doug Daniels Dev Blog ramblings from another game developer

3May/090

Building iPhone XCode Projects with Maven 2 and Hudson

I'm a Java developer who recently began doing some iphone development, and was pleasantly surprised to find out that Apple's XCode supports a command-line build tool called xcodebuild. xcodebuild will compile, run unit tests, sign and package your iphone or Mac OS X application with one simple execution of:

xcodebuild -alltargets

This can be easily integrated into an ANT build.xml script which allows to to then integrate into a Maven pom.xml which will allow you to manage your entire project from generating documentation and project website documents as well as publishing your application's build artifacts to release servers.

I setup a SVN + Apache server on one of my mac minis using Macports following these instructions.

http://www.sonzea.com/articles/subversion-trac.html

sudo /opt/local/apache2/bin/apachectl -k restart

Then I setup Nexus a local maven respository for deploying applications and managing third-party repositories and releases and Hudson a continuous integration server as a Mac OS X server daemon through launchd configuration.

The result is a nice development pipeline that blends well with my Java development experience and when I start developing more advanced applications I'll be be able to manage the build of Java servers and clients (Google's Android clients) as well as iphone (XCode objective-c projects).

In summary I setup the following development processes:

Next steps...

  • Continue research into high performance networking with a lean towards Java and iPhone
  • Apache MINA - High performance Java networking library
  • Apache Thrift - Facebook framework for creating language independent network protocols includes RPC client/server for a bunch of programming languages
  • Google's protobuf - Google framework for
  • Project Darkstar (and Dimdwarf))
Filed under: Uncategorized No Comments
1May/091

Nerd Scrapbooking (retrospective from a pack rat of moleskines and notebooks)

I decided to start a blog again to help keep myself motivated and get some side software development projects going that I keep meaning to get around to. I started going through some of my old notebooks and moleskine journals and realized I'm kinda of an obsessive note taker and collector of information.

I was going through a pile of legal pads that basically chronicled my short year at NCsoft (From my interview discussing the 3 spheres of powers of the Blighted Empire haha, to technical notes about converting the Guild Wars network protocol to a Java client, to model view controller design pattern notes on the last prototype project I was on at NCsoft).

Here's some other random crap I came across going through my old notebooks.

Google visitor badge
when I interviewed with them in Boston, MA, I didn't get the job but it was a really good experience I wrote about it on my old blog (which may still be available on facebook my original blog site got hacked a while ago). If you want the gory details you can read my post about the google phone interview process and producer consumer programming problem.

Apple iPhone Tech Talk sticker for Austin, TX, Mark and I went to Apple's hosted iPhone tech talk it was pretty cool, to short though (but it was free).

3D Tracking using wii remote I found some random diagrams and calculations I was doing trying to make my own 3D tracking system using a wii remote infrared camera, I was inspired by Johnny Lee's Head tracking demo .

Over the years I've tried coming up with the "perfect" note taking system and organizational system mixing digital note archiving systems with GTD style moleskine notebooks, but every time I lock myself into a organizational system I end up becoming paralyzed by the discipline and end up not being able to take any notes.

The best process I can come up with is a completely informal process of dumping random thoughts on a variety of notebooks and notepads that I keep available all over my room and office, that I can just start writing in whenever I want to dump thoughts out or mind draw out some diagrams.

Currently this is what I have setup:
8.5'' x 14'' Legal Pads - For some reason I crave this large format note pad and because it's not like other side bounded notebooks I feel like it's more informal format.
Large ruled Moleskine notebook - I really love the aesthetic of the moleskine notebook I try to keep it with me wherever I go (work, home, traveling) I think of it as a slightly more formal or permanent place for notes and ideas.

Small moleskine weekly planner - I use the weekly planner to try to keep a daily/weekly journal of my life it acts as a more chronological journal that I can look back on (or plan ahead with).
Evernote - I use evernote application to collect task lists (project tasks, shopping lists) I also really like it's hand writing recognition image importing (I just take web cam photos of my notebooks and it will archive my hand-written notes).


Google Docs - I like throwing quick notes and ideas onto Google Docs because I can access it anywhere I can get to the internet, I use it as a sort of digital brain dump and private blog.


Gmail - Email is probably my main starting point for my personal information sometimes I find it easier to just keep notes by sending myself emails and searching for it later.

Filed under: Uncategorized 1 Comment
7Nov/070

New Job NCSoft Austin – Texas here I come!

Great news!!

I’m going to be working for NCSoft, in Austin, TX as a 3D game programmer!! I’ll be working with the Java Monkey Engine technology on a super-secret project (You know how it is with video games).

NCSoft is a premiere developer of PC based MMO’s like Tabula Rasa, Guild Wars, City of Heroes, City of Villains.

Now the really interesting thing about finding this job is that I have no direct game development work experience (by day I’m a java programmer for a military contractor, by night a ninja monkey game programmer), but as a personal passion, I’ve been learning, prototyping and extending projects using the jMonkeyEngine technology, and so through the jME community I was able to get in touch with two of the creators of the Java Monkey Engine who work at NCSoft Josh Slack, and Mark Powell. Over the past year or so through discussion with Mark and Josh and vigilantly monitoring the NCSoft jobs site :) I noticed they had some open positions for 3D client developers with Java Monkey Engine experience!! I immediately sent my resume in, and was lucky enough to get the chance to fly down to Austin and interview with the team!

The interview went great, I met some incredibly creative and interesting people (I absolutely can’t wait to work with the team). I was nervous that because I didn’t have any direct game related experience that I’d lose to a candidate whose been in the industry, but I feel that my passion, plus my unique experience and involvement in the jME community helped tip the decision in my favor.

It’s going to be a little crazy for my personal life, because I quit my current job (gave them my 2 weeks notice friday), and I’m going to be packing all my stuff up and shipping down to Austin, TX in 2 weeks!! I’ll be leaving behind 90% of my family and friends (except for my friend Matt who plans on quitting his job, and joining me on the cross-country adventure by moving down to Austin with me and finding some job down there). I’m at a point in my life where I need the change, I want the adventure, I’m up for the challenge and plus this job is a dream come true for me! Also Austin is supposed to be an incredible place to live and I can’t wait to check out the music scene and apparently there’s a huge technology maker/hacker scene down there, I’m excited!

Once I’m down there I hope to continue writing about my experiences working for a game development company, and also some more ideas I have on how people from the larger “corporate” software industry can break into the game development industry. Especially with the unique technologies like the Java Monkey Engine which is going to require developers to have strong Java and computer graphics experience, which is very different from the traditional C++/OpenGL game technologies.

It’s going to be very interesting to be part of the jMonkeyEngine community as more commercial companies begin to utilize it, I’m hoping that companies will realize that the competitive business edge is not in the core game engine framework but in the unique proprietary gameplay and tools specific to the game built on top of that technology, and that by leveraging and contributing to open source standardized core game technology, they can create a cheaper higher quality end product across the industry. Then the industry won’t be driven by a select number of multi-million dollar blockbuster games, instead it will be driven by a massive selection of games which allows developers more freedom to develop creative and engaging gameplay and stories. (or at least that’s the idealists dream right :) )

Filed under: Uncategorized No Comments
2Sep/070

Choosing an Open Source License for Game Development

I was looking back on an old post I made on the javalobby.org forums and realized that it was a decently coherent dialog I had (with myself), so in the vein of being vain I decided I should publish it on my blog.

It basically debates the merits of 2 open source licenses the BSD (Pretty much pure freedom but no guarantee of contributions from users) and the GPL v2 with classpath exception (Very strict forcing core contributions to be forced open sourced, but allows users to “link” to the libraries without open sourcing) and how they could be applied to the development of open source game related technologies within the context of proprietary game development. You can read the old post here:
Choosing an open source license for a Java Game tools

I’m researching what open source license I should choose to use when developing common game development open source software, that I want people to be able to use within proprietary software. The catch is I want there to be some kind of incentive for the users to help contribute back to the core open source technology.

I’ve been playing around the with the Java Monkey Engine at http://www.jmonkeyengine.com .

In my research I’ve been reading up a little bit about Sun’s move to open source Java under GPLV2 with some of it being licensed under GPLV2+classpath exception.

The classpath exception allows developers to link to the compiled GPL’d software without GPL’ing their own software (a good thing if you’re developing a proprietary game):

http://www.gnu.org/software/classpath/license.html

Linking this library statically or dynamically with other modules is making a combined work based on this library. Thus, the terms and conditions of the GNU General Public License cover the whole combination.

As a special exception, the copyright holders of this library give you permission to link this library with independent modules to produce an executable, regardless of the license terms of these independent modules, and to copy and distribute the resulting executable under terms of your choice, provided that you also meet, for each linked independent module, the terms and conditions of the license of that module. An independent module is a module which is not derived from or based on this library. If you modify this library, you may extend this exception to your version of the library, but you are not obligated to do so. If you do not wish to do so, delete this exception statement from your version.

The GPLv2 + classpath exception, is almost identical to the LGPL, in that LGPL allows you to link to the compiled library without being blanketed by the GPL (IE: having to contribute your proprietary code). Although the wording in the LGPL is a little vague and has some exceptions that aren’t that clear in a Java environment(see section 5 of the LGPL “TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION”). The classpath exception makes the intended use very clear.

I’m trying to decide what type of license to use when releasing source code that I want to be able to use in a proprietary closed source game.

The real conflict is that, I could open source my game libraries using a BSD license which gives you the most freedom to modify and redistribute the code anyway you like. The problem with that is you have to rely on the developer community of that library to want to contribute back to the source code, otherwise they could just grab it make awesome modifications and keep it for themselves (which is sometimes what you want to do anyways!).

I think with a project like jMonkeyEngine BSD will work because, the engine is the core functionality for game development and benefits all game developers, it’s not like it’s the specific code that implements a first person shooter, or a massive multiplayer RPG. This combined with a strong community, promotes people to contribute back to the jME open source code base.

The problem with using a BSD license for something like say a game scripting framework is that I would want people to be able to use it as they will (open source, proprietary), but I want some way that kind of “forces” users to contribute back any changes they make to the core scripting engine (but still allow them to utilize the library in their proprietary projects). In this sense I think the GPLV2 license with the classpath exception makes the most sense.

I think maybe even jMonkeyEngine could be released under this license, because it still allows proprietary developers to make extensions and add proprietary features to jME (they can still extend, and implement any class/interface in the compiled jME), but it forces them to contribute back any modifications/improvements/bug fixes they make to the core engine.

It’s all a trade off…BSD gives you all the freedom you could want proprietary and open source but provides no mechanism to ensure that the community receives all the available improvements/modifications that are made with that core technology. While the GPL license allows the freedom to develop proprietary software based on the compiled engine, but if they modify/improve the core (not simply link to it), then they are forced to contribute that back.

Another cool thing to do with proprietary games, is to release them under GPL after they’ve reached their commercial “end of life” like John Carmack has done with the Quake engines.

Another conflicting thing that came up with Sun’s open sourcing, is the fact that they still somehow maintain the right to make commercial releases from the GPL code, even though it is GPL’d because you sign the SCA you and Sun have dual-ownership of your contributions?? I guess this is something to do with the fact that they are the “original” copyright holders so they can change the license at will?

I might be getting this all wrong, but I think companies like mySQL have similiar dual-licensing schemes, where you have to use the commercial license for proprietary software, or you have to use the open source GPL license which means you must contribute your changes back to them??

Which implies that they can then take your, forced to contribute changes and sell them for a profit?? This seems like a major unethical loop-hole in the dual-licensing GPL system? Can anyone clear up how that makes any sense?

I guess the only thing I can think of is that the creator of the project is a copyright holder and can license the source code any way they want, and then someone who uses the system is a contributor and only has the rights of the license they received the source code with? What does that mean for the contributor who wants to contribute their modifications back, but doesn’t want their changes to be exploited by MySQL’s commercial license?? Is this where open source project forks come into play? I’d assume they have to have complete control over what code is GPL’d and what code is their proprietary commercial code? Seems like that creates the exact thing GPL wants to protect against.

Here’s some good open source/free software licensing sites:

Free Software Foundation - Licenses
Why you should not use LGPL

There is a post on the Java Monkey Engine forums about this topic, I just wanted to get a wider Java community opinion also.

Filed under: Uncategorized No Comments
2Sep/070

Scripting in the jMonkey Engine

I wrote up a quick wiki page about a Scripting framework I put together for the Java Monkey Engine. I should have posted this months ago, but I thought I’d actually polish it off beyond this example prototype. I’m hoping if I publish it now, I’ll get some personal motivation to actually start some more serious development on it.

jMonkey Engine Scripting

Filed under: Uncategorized No Comments
29Aug/070

Scripting the Processing.org Framework

I wrote up an article on my wiki:

Scripting the Processing Language


I created a PAppletScriptable Applet that allows you to on the fly write javascript that will control a processing application.

Try it out here (you’ll need JDK 6 and I’ve only tested it on Firefox so good luck, i’ll update it soon to be more ‘polished’): PAppletScritable test

Filed under: Uncategorized No Comments
14Aug/070

Google Interview

Last week I had a technical phone interview with an engineer from Google’s Mountain View office. He presented me with the problem description for a classic producer consumer concurrent programming problem. This being the 3rd freakin’ time I’ve interviewed with them in the past year, I wasn’t to nervous this time, because I’d basically already given up on them and was just doing this because they contacted me to interview with them again.

Long story short I nailed the producer consumer problem IMHO, possibly because as a side hobby I’ve been reading up on concurrency in Java. The interview lasted exactly 45 minutes, and at the end the interviewer seemed pleased with how it went. I got a call back the next day saying they wanted to setup an on-site interview at the Boston area Google office in Cambridge. I’ll be going there next wednesday 8/22 for a 2 hour on-site interview. I’m pretty pumped about this seeing as it’d be my dream job to work for Google and have a chance to work on and even start some really innovative projects.

The technical momentum and community that Google has created is simply amazing. They’ve been able to create an environment where you get to work on the most interesting and sometimes esoteric computer science problems, and yet those problems have practical applicability in Google’s mission to make all of the worlds information universally accessible.

If you want to read more about the interview I wrote up a detailed article on my wiki:

Google Interview Details

Filed under: Uncategorized No Comments
11Mar/070

JDDaniels Open Source Java project created

I created a project on Google’s code hosting site, to host common code that I create such as tutorials, little scripts and examples. I feel like I code up stuff and then save it in some obscure project on my hard drive and always have to dig around for it when I want to grab ideas from it.

So instead I’m going to try and organize my code in a subversion repository, and release it to the world under the BSD license.

JDDaniels project @ code.google.com

JDDaniels wiki @ Event Horizon Games Wiki

The first thing I’ll be doing is posting up my Project Euler solution for the first problem I attempted. I’m also going to post up any jMonkey Engine example/tutorial code I come up with.

I think this will be a good place to persist any code tutorials I write up on this blog or on my wiki.

Filed under: Uncategorized No Comments
22Dec/060

Concurrent and Multithreaded Programming Articles

I just read an interesting post on Mark Powell’s blog (he’s one of the creators of the Java Monkey Engine).

He provided some very interesting ideas about concurrency and scalability of game engines seeing as we’re moving towards systems with multiple cores, and we should not be constrained to coding for a fixed number of cores, we should be scalable to N cores.

I summarized a list of articles including Mark’s post (and shamelessly some of mine), on the jME wiki and Event Horizons wiki (I know its duplication, but I’ll try to keep them in synch):

  • jME Wiki - Concurrency and Multithreading in Game Design
  • Event Horizon Games Wiki - Concurrency and Multithreading in Game Design
  • Check it out it’s a good read:

    jME: Multithreaded Madness

    I posted a comment to his blog, but then I discovered this nifty Trackback feature, so I’ll repost my comment here (I’m kinda using this as an experiment using trackback URI’s):

    Here’s my comment to his post:

    Mark,

    Very interesting post!

    I agree with you that the real challenge for the future is not to find a few parralel tasks in your game engine (physics, collision detection, AI) because that limits your concurrent scalability, you need to find tasks that are scalable to n cores.

    Another interesting feature of some of the java.util.concurrent collection objects such as the ConcurrentHashMap is that they are not only thread safe but their performance is concurrently scalable. They are scalable because they use a concurrent finer grained locking scheme called lock striping, this allows the ConcurrentHashmap to only lock portions of its bucket so that multiple threads could write to it safely (The book Java Concurrency in Practice has a whole section dedicated to Concurrent Performance and Scalability it’s a very interesting read check out their example of a StripedMap).

    Another lock-free idea that provides a high capability for concurrency is Software Transactional Memory. The idea is very similar to atomic transactions in the database world, STM optimistically assumes that any concurrent transaction that is made on shared memory is going to be done without another thread trying to access that shared memory. The key idea is that it makes the concurrent code developer not have to write explicit synchronization to perform actions, but once that threads transaction is submitted then the application will validate that no other thread has accessed that shared memory, and then commit it. If it fails then the transaction is retried, in real practice this transaction retry mechanism has proven more efficient because actual conflicts are rare, rather than having threads explicitly acquire locks per transaction. In the end STM provides a way of creating a highly concurrent application (it will scale to N processors/cores).

    Right now I’m just researching a bunch of concurrency topics to get an idea of the whole range of ideas out there, I hope to be involved in the future direction of jME’s concurrent architecture.

    I started an article on the jME wiki to summarize any interesting articles about concurrency and multithreading in game design:
    * Concurrency and Multithreading in Game Design

    It’s a mirror of the one I have on my developer wiki, I’ll try to keep the 2 in synch.

    Filed under: Uncategorized No Comments
    24Nov/060

    Simulated Concurrent Game Scripting using coroutines

    I’ve been researching some topics in concurrent programming as it applies to game design, and figured I’d post some of the more interesting pieces.

    The article, Multithreaded Game Scripting with Stackless Python is an extremely detailed article that discusses using Stackless Python to provide a game scripting framework that allows game designers to write actor/entity code in a more intuitive independent process oriented approach, as opposed to the standard time-slice update() every frame approach. I’ve been trying to come up with a way to integrate this functionality into jME (Java Monkey Engine), and am taking a look at the new Java 6 Scripting Engines to see if any of the available languages support coroutines (the basic elements of cooperative multithreading). The big idea is *NOT* to provide a true multithreaded framework, in the sense that these software only threads will be run in a single-threaded simulated cooperative threaded environment.

    The basic idea is to allow game designers to write scripts that are run in a non-preemptive (cooperative) simulated multithreaded environment, by simulating each script in the game system as if it were a concurrent processes.

    Scripting languages like Lua, and Python support coroutines which in a nutshell allow you to write functions that can yield their execution and then resume that execution at a later time. This provides a great way to simulate a multithreaded environment without the overhead of real operating system level threads, and with the benefit of being able to schedule your threads to be executed sequentially (eliminating any synchronization issues). If this all seems very abstract don’t worry it is, I’m hopefully going to be provide more concrete implementation details soon (The Game Programming Gems series have some great articles about using Lua to create a concurrent scripting framework).

    Check out a tutorial on Lua’s coroutines to get an idea of how they work:
    Lua Coroutines Tutorial

    The real benefit of all this simulated abstraction is that you can write code like this:

    Concurrent scripting close() event method:

    def close(self):
        try:
            self.setVelocity(self.mMoveVelocity)
            self.sleep(self.mCloseMoveTime)
        except CCollition, col:
            print “Oops.. sorry”, col.name
    
        self.setVelocity(CVector3.ZERO)

    Instead of the time interpolated update() state checking approach:

    void update(t_actor *actor,double timedelta){
        if(actor->state == CLOSING){
            Cvector3 pos = actor->origin;
            pos = pos + actor->velocity * timedelta;
            if(pos == restpos){
                actor->state = CLOSED;
                return;
            }
            actor->origin = pos;
        }
    }
    Filed under: Uncategorized No Comments