[WaeV]

With the help of Modzy's Open Halo Parser and some specific help from Oxide, I've begun breaking into programming for Halo. I've been prototyping in Python, but I've also found my Python code to be a useful exploration tool for the map's structure.

I didn't know what to call it or how to release my code, so I settled on "PyHaPa" (Python Halo Parser) and the GPL. You'll need Python 2.7 to run this. At the moment I'm focusing on Halo 1 PC maps only, but I think CE maps work as well.



PyHaPa 1.1 Download



Invocation examples

(You would put these code examples in a .py file in the same directory as HaloMap.py, HaloTag.py, and BinaryFile.py.)

Displaying the general map information

from HaloMap import HaloMap

map0 = HaloMap('C:\\bloodgulch.map')
map0.load()
map0.display()

Output:

Spoiler

+--------------
| Head String: head
| Game Version: 7
| Map Size: 14366196
| Index Offset: 0x851974
| Metadata Length: 5643392
| Map Name: bloodgulch
| Version: 01.00.00.0564
| Map Type: 1
+--------------
| Memory Offset: 0x40440028
| Map Magic: 1069475468
| Base Tag ID: 3782475776
| Map ID: 0x5e2af81e
| Tag Count: 2408
| Verticie Count: 447
| Verticie Offset: 0x173000
| Indicie Count: 447
| Indicie Offset: 0x68bd84
| Model Data Length: 7203188
+--------------

Displaying all tag names

from HaloMap import HaloMap

map0 = HaloMap('C:\\bloodgulch.map')
map0.load()
map0.displayAllTags()

Output: Way too long to post all of it, but the tags are displayed like so:

Spoiler

[scnr]levels\test\bloodgulch\bloodgulch
[sky ]levels\test\bloodgulch\bloodgulch
[mod2]sky\mp clear afternoon\mp clear afternoon

etc...

Using logic to do stuff

from HaloMap import HaloMap

map0 = HaloMap('C:\\bloodgulch.map')
map0.load()

for tag in map0.tags:
    if tag.ident == map0.base_tag_ident:
        tag.display()

Output:

Spoiler

levels\test\bloodgulch\bloodgulch
+---------
| First Class: scnr
| Second Class: ÿÿÿÿ
| Third Class: ÿÿÿÿ
| Ident: 0xe1740000L
| Name Memory Offset: 0x40452d28
| Name Offset: 0x86469c
| Meta Memory Offset: 0x4046ed34
| Meta Offset: 0x8806a8
| Indexed: 0

I don't know how references work yet, but that's my next target once I return to this project.

[TCK]

Great progress man. I'm thinking of getting started on more in-depth coding by making some program(s) for Halo, might ask around here for various answers since the site is full of those who've done so already.

[WaeV]

Shameless self-plug here, but I put extra effort into documenting my code, so it's a good place to start even if you choose a different language.

You're right though - there ARE a lot of Halo programmers here. If any of them are reading this right now, any pointers on reading a tag's references?

[XZodia]

Python? Really? Ugh....

What do you mean by pointers on reading tag references? (I'm sure there's a pun in there somewhere...)

[WaeV]

XZodia, on 08 May 2011 - 12:09 AM, said:

Python? Really? Ugh....

What do you mean by pointers on reading tag references? (I'm sure there's a pun in there somewhere...)

Yeah... I only did this in Python as a prototype. I've written a simple little program in C++ which, given a map, prints all the tag names in order.


Heheh, either way you interpret it, help would be cool. At the moment I only have the vaguest of ideas on how tags are referenced. It appears to be 16 bytes per reference: the referenced tag's class, the offset to the referenced tag's name, 4 bytes of bufferspace, and the referenced tag's ident.

If that's true, then what I need to figure out is how to parse any given tag for references. They don't seem to be evenly spaced.

[XZodia]

That sounds about right, they are evenly spaced but I can't recall the details...

[WaeV]

I've been following through the 21 Code Kata to get more acquainted with JRuby.

Also, this is a pretty good article on programming languages. What do you think of his opinions? At the moment I'm interested in looking into JRuby, Scala, and Clojure - all JVM languages which can integrate pretty well with Java if need be. As much as I may dislike Java, it's pretty standard in the industry right. I figure learning languages which can interact with it is a decent move.

This post has been edited by WaeV: 14 May 2011 - 06:07 PM

[XZodia]

I've never heard of any of those languages...also java is shit.

[kornman00]

Multiple language interop is a hindsight feature of JVM. CLR (ie, .NET) was developed from the ground up with it in mind, and doesn't suffer from the same crap that was introduced by Java. There are loads of languages which you can use over the CLR, (Scala isn't on the list, but it was built around Java to begin with).

And before anyone goes "blah blah, .NET isn't cross platform", you can stuff it, because there's Mono (not to be confused or pronounced in the same way as the "kissing disease"), which allows you to run on Mac and Linux (plus has flavors for iOS deployment too).

I'd rather crawl back to working for the gov't again than spend my days programming with Java or on the JVM.

Nothing personal against you of course, WaeV

[Patrickssj6]

No point in using Java here. Ruby, Perl and Python etc. are all scripting languages. They are designed to do small tasks fast and easy. They are often used for writing exploits.

I made the mistake back then not listening to Korn and started writing in VB and not in C# in the beginning If you know C# / C++ you are set for almost (>99%) everything.

Nice progress though. It is very good to understand everything.

[WaeV]

kornman00, on 14 May 2011 - 06:40 PM, said:

Multiple language interop is a hindsight feature of JVM. CLR (ie, .NET) was developed from the ground up with it in mind, and doesn't suffer from the same crap that was introduced by Java.

JVM bytecode isn't restricted by the same shortcomings as Java. Additional support for dynamic languages on the JVM is in the works, and closures are on the way as well. (Still waiting on that tail-call optimization, though...)

kornman00, on 14 May 2011 - 06:40 PM, said:

And before anyone goes "blah blah, .NET isn't cross platform", you can stuff it, because there's Mono (not to be confused or pronounced in the same way as the "kissing disease"), which allows you to run on Mac and Linux (plus has flavors for iOS deployment too).

Yeah, uh-huh. There's a fair handful of features which haven't been implemented into Mono yet. Everywhere I look for information on Mono, I see "this may not work properly in Mono" disclaimers.

kornman00, on 14 May 2011 - 06:40 PM, said:

I'd rather crawl back to working for the gov't again than spend my days programming with Java or on the JVM.

Agreed with the bit about programming in Java. Awful stuff. The JVM is nice, though. CLR may be fine on Windows, but for cross-platform, JVM is the way to go.

kornman00, on 14 May 2011 - 06:40 PM, said:

Nothing personal against you of course, WaeV

Oh of course, lol.

Patrickssj6, on 14 May 2011 - 06:50 PM, said:

No point in using Java here. Ruby, Perl and Python etc. are all scripting languages. They are designed to do small tasks fast and easy. They are often used for writing exploits.

Perl and perhaps Python I concede, but even so - don't be so hasty to write them off as just "scripting languages". Twitter's backend used to be completely implemented in Ruby, before they switched to Scala. Ruby on Rails is most commonly used to write web apps in a short amount of time, but

Patrickssj6, on 14 May 2011 - 06:50 PM, said:

Nice progress though. It is very good to understand everything.

Thanks! It's fun, too.

This post has been edited by WaeV: 14 May 2011 - 11:35 PM

[SnipeYa]

Just a suggestion but you should make a program that does halo 1 bitmaps similar to the way eschaton does but without the pixle-ation and that be the only thing it does , i would love you forever and always for an application like that.

please oh please someone make an application like that!

[kornman00]

WaeV, on 14 May 2011 - 11:24 PM, said:

Yeah, uh-huh. There's a fair handful of features which haven't been implemented into Mono yet. Everywhere I look for information on Mono, I see "this may not work properly in Mono" disclaimers.

Let me guess, you were browsing code which dealt with one or more of the following: WPF, WCF, anything non-BCL, anything else .NET 3.5 or higher or something dated to the previous decade? The first two are pretty obvious as to why they wouldn't potentially work with Mono since they're prefixed with Windows (expect they do plan to implement parts of WCF), however, the Mono devs started work off the open source of .NET 2 (and today MS is at .NET 4), so they can't exactly keep up with the advances of MS in a timely manner.

You can do anything in Mono that you can do in .NET, which isn't exotic. I'd port some of my code if Mono had C++/cli support, but like say, that's an exotic part of MS's .NET.

Just because people put up the disclaimer, doesn't mean it doesn't work. Also, WPF may be a feature of .NET 4, but I'd hardly call it a useful feature of cross platform minded code. Mono provides technical information on their various technologies, including the related MS-based tech (eg, WinForms, which is supported up to 2.0)

[WaeV]

So what's left?

What can I use to make a native-looking, well-performing application?


E: Although for the record, I'm looking into JVM languages partially as a career move - I can develop small applications, yes, but in the future I can also work with companies utilizing the JVM.

This post has been edited by WaeV: 15 May 2011 - 05:05 AM

[Patrickssj6]

WaeV, on 15 May 2011 - 04:48 AM, said:

So what's left?

What can I use to make a native-looking, well-performing application?

I don't know about you but imo Java is far from native-looking and well-performing

WaeV, on 15 May 2011 - 04:48 AM, said:

E: Although for the record, I'm looking into JVM languages partially as a career move - I can develop small applications, yes, but in the future I can also work with companies utilizing the JVM.

How about learning C# which has a very similar syntax to Java. You can easily switch from the one to the other. Add C++ to your repertoire and you can forget about Java.^^

[SnipeYa]

Patrickssj6, on 14 May 2011 - 06:50 PM, said:

No point in using Java here. Ruby, Perl and Python etc. are all scripting languages. They are designed to do small tasks fast and easy. They are often used for writing exploits.

I made the mistake back then not listening to Korn and started writing in VB and not in C# in the beginning If you know C# / C++ you are set for almost (>99%) everything.

Nice progress though. It is very good to understand everything.

what is so bad about VB

[xbox7887]

SnipeYa, on 15 May 2011 - 10:24 AM, said:

what is so bad about VB

It better accommodates the careless programmer. Plus, it just looks like ass

[neodos]

xbox7887, on 15 May 2011 - 05:56 PM, said:

It better accommodates the careless programmer. Plus, it just looks like ass

Noes! okay yeah probably, though it does the job for simple apps

[WaeV]

Patrickssj6, on 15 May 2011 - 10:13 AM, said:

I don't know about you but imo Java is far from native-looking and well-performing

http://keithlea.com/javabench/

The JVM is certainly well-performing. Although having looked into it just now, it seems that C++, .Net, and JVM languages are all "close enough" in speed that it makes little difference. Sometimes one is faster, sometimes others.

As for look-and-feel, there are bindings for all types of GUI toolkits so I guess that's a moot point as well.

Patrickssj6, on 15 May 2011 - 10:13 AM, said:

How about learning C# which has a very similar syntax to Java. You can easily switch from the one to the other. Add C++ to your repertoire and you can forget about Java.^^

Mm. I think at this point that both the JVM and .Net are good choices. There is Java (ew), JRuby, Scala, and Clojure on the one hand, and C#, F#, and the Iron~ languages on the other.

[kornman00]

If you're wanting to be cross-platform, wouldn't you want a consistent look-and-feel, not a "native" (and thus, more specific to a platform) one?

My main disgusts with Java have always been: people touting it as "pure-OO" when it's not (primitives in Java are not objects), no operating overloading (please, I'm not a fucking child, I don't need some idiot to "protect me from myself"), no support for value types (this actually goes with the first point), no support of unsigned integers (really? come on), and the overhead needed to develop with Swing. There's more, but those are my main gripes I've always expressed. Not sure how much of it applies to the JVM.

How many companies/future possible employers exclusively use the JVM? I've never heard of any kind of study, so I'm curious as to what your findings are for current companies.