ROM Hacking?

[mfgreth]

Has there been any effort to hacking this game in the past, outside of translating? It would be cool if someone produce a toolkit to do so, so we could produce fan-created fields/challenges. Thoughts?

(I'm not proposing to do this myself, I'm not much of a programmer, and my only experience ROM hacking was with SMW and Chrono Trigger... like everyone else basically).

[Nicole]

Uhh, there were a few efforts here and there. There were a couple efforts to make a level editor, though they didn't come to pass. I dug up a Japanese field viewer at some point, which seems like it was meant to become an editor at some point, though that never happened.

There's also a level select hack I got from Electric Lady, though it's kind of buggy. I also took a small crack at reverse-engineering the game myself, though I eventually lost interest for a while since reverse-engineering assembly is rough-going. The timer routine was the main thing I figured out, in any case -- I didn't try braving the physics or map systems. I wrote some notes while doing so, but they wouldn't be very intelligible to anyone but myself as they are right now.

As for documentation regarding the internals of the game, there's a few scattered bits of information, as well as the wiki for one failed attempt at a level editor... I dunno if that's still around. Anyway, I'll try and dig up some of this stuff tomorrow.

[KawaseFan]

Hopefully one day a level editor is made, that would be fantastic.

[Nicole]

Here's the field select patch. This patch should be applied to a Japanese headerless ROM. It's possible that an English-patched ROM could work as well, but there's not much point since this patch breaks the replay and pause menus.

Here's the collection of Japanese resources I found. It includes some Japanese notes on some of the game internals, a field viewer (plus source code!), and a small, somewhat pointless patch that basically just lets you complete the game instantly.

Here's a small linkdump of some resources I found:
http://wiki.superfamicom.org/snes/show/Umihara+Kawase
http://acmlm.kafuka.org/board/thread.php?id=6331
http://ndirt.com/umihara/doku.php?id=start

I've got a feeling that the Japanese stuff in particular will prove useful for actually creating a level editor, but the fact that most of it's in Japanese makes things somewhat tedious.

[mfgreth]

I've got a feeling that the Japanese stuff in particular will prove useful for actually creating a level editor, but the fact that most of it's in Japanese makes things somewhat tedious.

Nice finds!

It's true, it will be very tedious. But it's always worth looking into, anyways.

[KawaseFan]

It's a real shame that all that research has been done with not a lot of result so far, but at least that information and code is still around for anyone who might be interested in the future.

I recognise a username from some of those links - also someone apparently involved with the wiki at ndirt.com - as someone who got to the point of actually offering to pay someone to make a level editor, and that doesn't seem to have resulted in anything, so I don't know what it'll take for it to happen.  Maybe it's that hard to do?  Maybe €100 wasn't enough of an offer?

[badlose]

I've talked a couple people throughout the years who were working on hacking SFC Umihara Kawase, they never got too far. Minor stuff like changing the timer variable, or jump height variable. I'd be most interested in the formula for calculating the line physics, as much as I'd love a level editor someday.

[Nicole]

Well, I'm looking over the field viewer source code right now. Can't promise anything right now, obviously, but it does seem to me that if we can read the map data with this code, writing shouldn't be that hard, assuming there's no compression (which there doesn't seem to be). I've gotta get a handle on what the code is doing first, though.

[Nicole]

Doing a little, er... experimentation.

[Nicole]

Working on translating the Japanese notes, because there's a lot of information in there.

I've found out that the field data is in fact compressed, and explains what Naulahauta was seeing with seemingly random "00"s and other byte combinations sprinkled in the data--the fact that he could modify the data easily at all was because of sheer luck that the tiles he wanted to modify were uncompressed.

This doesn't mean that modifying fields is impossible or anything, but it is made somewhat more difficult by the fact that the new field data must fit in the space the original field fit in. This could possibly be sidestepped with ROM expansion, though at the moment I'm not sure how stable the ROM would be, or if some code would need to be altered.

That being said, the compression algorithm used to store the original fields doesn't seem terribly efficient, so it may be possible to squeeze altered fields in easily. It's hard to say at this stage.

EDIT: Though of course, just flat out removing all the original fields and adding in new fields would be no problem at all.

[badlose]

Pretty exciting stuff. Keep us updated, or let us know if we can help in any way.

[Nicole]

Code Select Expand

    Addr     FieldType No. MusicType ... HW WtrType  Time  Next
00: $98:a10e Riverside F00 Riverside     f0 01 02 e6  5:00 F01 F01 F01 F01
01: $99:b08a Riverside F01 Riverside     f0 01 02 e6  5:00 F02 F02 F02 F02
02: $97:b403 Riverside F02 Riverside     e8 01 02 e6  5:00 F03 F05 F05 F05
03: $97:d58e Seaside   F03 Seaside       f0 01 03 e6  4:00 F04 F04 F04 F04
04: $85:f000 Seaside   F04 Seaside       f0 01 03 e6  4:00 F39 F39 F39 F39
05: $93:c49c Riverside F10 Riverside     f0 01 02 e6  4:00 F09 F09 F09 F09
06: $8b:e0ff Seaside   F06 Seaside       f0 01 03 e6  4:00 F07 F07 F07 F07
07: $8a:e485 Seaside   F07 Seaside       f0 01 03 e6  6:00 F08 F08 F08 F08
08: $9b:d1e3 Riverside F08 Riverside     e8 01 02 e6  4:00 F10 F10 F10 F10
09: $9a:ad8c Riverside F11 Riverside     e8 01 02 e6  5:00 F08 F20 F12 F12
0a: $92:f0fd Seaside   F14 Seaside       e8 01 03 e6  4:00 F11 F11 F11 F11
0b: $98:908a Waterfall F15 Waterfall     f0 01 02 e6  4:00 F55 F59 F59 F59
0c: $9a:9e6e Waterfall F12 Waterfall     f0 01 02 e6  5:00 F63 F63 F63 F63
0d: $99:dfc9 Waterfall F17 Waterfall     f0 01 02 e6  5:00 F14 F28 F28 F28
0e: $99:8000 Seaside   F20 Seaside       e8 01 03 e6  5:00 F15 F15 F15 F15
0f: $9a:e6cd Seaside   F21 Seaside       e8 01 03 e6  4:00 F16 F16 F16 F16
10: $95:e923 Mt.Stream F22 Mt.Stream     f4 01 02 e6  4:00 F51 F22 F22 F22
11: $8c:f27d Seaside   F26 Seaside       e8 01 03 e6  4:00 F52 F50 F50 F50
12: $91:e749 Riverside F18 Riverside     e8 01 02 e6  4:00 F51 F51 F51 F51
13: $93:ecae Riverside F38 Riverside     e8 01 02 e6  4:00 F54 F54 F54 F54
14: $97:a2fa Seaside   F30 Seaside       f0 01 03 e6  4:00 F49 F60 F60 F60
15: $8b:f3da Seaside   F45 Seaside       e8 01 03 e6  5:00 F45 F45 F45 F45
16: $9a:cb44 Mt.Stream F24 Mt.Stream     f4 01 02 e6  5:00 F61 F43 F43 F43
17: $97:f320 Mt.Stream F35 Mt.Stream     f4 01 02 e6  5:00 End End End End
18: $92:e10e Riverside F05 Riverside     f0 01 02 e6  4:00 F06 F06 F06 F06
19: $96:f214 Wharf     F48 Wharf         e8 01 03 e6  5:00 F62 F62 F62 F62
1a: $9a:8f3a Wharf     F47 Wharf         e8 01 03 e6  5:00 F40 F40 F40 F40
1b: $99:ef2e Ocean     F52 Ocean         e8 01 03 e6  5:00 F48 F48 F48 F48
1c: $9a:d927 Mt.Stream F42 Mt.Stream     f4 01 02 e6  6:00 F18 F46 F46 F46
1d: $99:9032 Ocean     F34 Ocean         e8 01 03 e6  5:00 F38 F38 F38 F38
1e: $98:f1cd Ocean     F50 Ocean         e8 01 03 e6  5:00 F58 F58 F58 F58
1f: $99:a063 Riverside F43 Riverside     e0 01 02 e6  5:00 F60 F60 F60 F60
20: $8e:f015 Seaside   F57 Seaside       f0 01 03 e6  5:00 End End End End
21: $91:f5f3 Ocean     F55 Ocean         e8 01 03 e6  5:00 End End End End
22: $9a:bc97 Mt.Stream F33 Mt.Stream     f4 01 02 e6  5:00 F53 F53 F53 F53
23: $96:bccd Mt.Stream F31 Mt.Stream     f4 01 02 e6  8:00 F57 F57 F57 F57
24: $9a:8000 Mt.Stream F23 Mt.Stream     f4 01 02 e6  5:00 F37 F37 F37 F37
25: $98:d252 Mt.Stream F28 Mt.Stream     f4 01 02 e6  4:00 End End End End
26: $98:e293 Mt.Stream F37 Mt.Stream     f4 01 02 e6  4:00 F22 F22 F22 F22
27: $97:c4d5 Waterfall F41 Waterfall     f0 01 02 e6  5:00 F43 F43 F43 F43
28: $96:ceed Waterfall F16 Waterfall     f0 01 02 e6  5:00 F13 F13 F13 F13
29: $96:aa70 Waterfall F29 Waterfall     f0 01 02 e6  5:00 F44 F44 F44 F44
2a: $8f:ec7e Wharf     F46 Wharf         e8 01 03 e6  5:00 F41 F41 F41 F41
2b: $87:f000 Wharf     F51 Wharf         e8 01 03 e6  5:00 F62 F62 F62 F62
2c: $89:f8e8 Mt.Stream F40 Mt.Stream     f4 01 02 e6  4:00 F38 F38 F38 F38
2d: $99:c092 Riverside F49 Riverside     f0 01 02 e6  6:00 F42 F42 F42 F42
2e: $9b:994a Seaside   F25 Seaside       f0 01 03 e6  4:00 F18 F56 F56 F56
2f: $90:f69c Wharf     F56 Wharf         e8 01 03 e6  6:00 F47 F47 F47 F47
30: $9b:b9c7 Ocean     F36 Ocean         e8 01 03 e6  8:00 F42 F36 F36 F36
Doing some testing with reading data from the ROM. This is a trimmed and formatted version of the field table, which holds information like field numbers, tileset, music, and water height. As you can see, the order of these table entries doesn't really correspond to the actual field numbers, and thus probably signifies the order in which the fields were created.

[KawaseFan]

Pretty interesting!

So - and I know pretty much nothing about ROM hacking, so please forgive me - but in theory the 'next' fields could be changed to lead to whichever field/s you want, or the time limits could be changed to be much quicker, and things like that, right?

[Nicole]

Yep, everything in this table is very easy to modify. There's also a large amount of blank space after it, and the length of the table is not hardcoded, so adding more entries is also very easy.

[KawaseFan]

That sounds pretty awesome!

[mfgreth]

These are some very exciting developments, Nicole!

[Nicole]

Making some progress on decompressing field data, but there seems to be some bugs in my code. Not sure if that's my fault or if the description in the notes is incorrect, but I'll look into it more. At the very least, the length of the data is coming out correctly, even if the data itself isn't, so I think I have an idea what the problem might be.

[Nana]

At the very least, that sounds like making a patch where you play every level in the game consecutively would be trivial. That of course, does break the spirit of the game, but if you're already an expert, then it would be fun to go through every level in the game in one playthrough.

[Nicole]

Yeah, a patch like that would be easy to make.

But actually, looking at this table a bit more... It seems I made a mistake in how this should be read. Like this line here:

Code Select Expand

    Addr     FieldType No. MusicType ... HW WtrType  Time  Next
05: $93:c49c Riverside F10 Riverside     f0 01 02 e6  4:00 F09 F09 F09 F09
F09 doesn't actually exist. In fact, what Next actually stores is the entry number in the table, not the actual field number. In this case, the ninth entry is:

Code Select Expand

09: $9a:ad8c Riverside F11 Riverside     e8 01 02 e6  5:00 F08 F20 F12 F12
And indeed, the field after F10 is F11. I'll go and fix the table in that post.

EDIT: Or... I was going to, but...

Code Select Expand

19: $96:f214 Wharf     F48 Wharf         e8 01 03 e6  5:00 3e 3e 3e 3e
There is no entry 3e, but the next field should be:

Code Select Expand

2f: $90:f69c Wharf     F56 Wharf         e8 01 03 e6  6:00 2f 2f 2f 2f
This is getting kinda perplexing...

[Nicole]

Tracing through the assembly code to try and figure out how the Next entries work, but it seems really bizarre so far... I dunno if it has something to do with the little tutorial sequences that show up before fields or what, but I'm looking into it. So far there seems to be two other tables it consults...

[KawaseFan]

That about the tutorials sounds like it would make sense.

[Nicole]

Yo, just a heads-up, this topic's been at a bit of a standstill for a while, but that's because I've been caught up in some other things recently. I'll get around to working on this again, but it might be a week or two.

[KawaseFan]

Not a problem at all.  The work you've already put into this is much appreciated by myself and I'm sure everyone here.

[mfgreth]

Nicole, you are absolutely awesome!

[badlose]

Unused/different behavior enemies from the /vr/ thread:

https://www.youtube.com/watch?v=4SLTEzKCmPs
https://www.youtube.com/watch?v=mgMI4AApyLE
https://www.youtube.com/watch?v=XjPet34iXMg
https://www.youtube.com/watch?v=vpj3WU_Y2Pw

[KawaseFan]

I really like the catfish in particular.  I wonder why they weren't used.

[Nicole]

Oh, yeesh. I didn't realize I'd said 1-2 weeks here. Guess I was a bit too optimistic.

The motherboard on my PC ended up crapping out on me, so although I haven't lost any work or anything, I'm not quite able to work on this at the moment. Not sure when I'll be able to again, but hopefully it's not too long. At the very least I'll try to put together some documentation of the stuff I've already figured out.

[KawaseFan]

Sorry to hear about your motherboard.  At least you didn't lose any data, though, that's the most important thing.

[Alc]

Unused/different behavior enemies from the /vr/ thread:

https://www.youtube.com/watch?v=4SLTEzKCmPs
https://www.youtube.com/watch?v=mgMI4AApyLE
https://www.youtube.com/watch?v=XjPet34iXMg
https://www.youtube.com/watch?v=vpj3WU_Y2Pw

These are fascinating. Is Verneri here? I had conversations with him about his level maps a few years back, but this is new stuff.

He also did this recently:

https://www.youtube.com/watch?v=3Z9QRSCrj0I

which is incredible. I'm guessing it must have taken some kind of emulator hack - lots of work no doubt, anyway. Well worth a look.

[Naulahauta]

I had no idea this forum existed until now – I only stumbled here upon searching "Rom hacking Umihara Kawase" yesterday.
I have a lot to say. First off, Raccoon Sam, Naulahauta and vervalkon, are all the same person, which is me, Verneri Kontto. I frequent Twitter and /vr/ these days more than any forums, and you will probably find more content under the vervalkon handle these days and in the future.

My contributions include the unused enemies mentioned earlier, comprehensive Sprite rips, Map rips and the Atlas vid. I hereby give permission for Kawasefan.net to use and host all of the media.
I also made a ROM hack that links all levels together, removes the 30-minute rule and implements the unused enemies in some levels, but I made that as a /vr/-only novelty and I no longer have the link. I never even made a patch, I simply posted the .smc. Someone might have it though.

About ROM Hacking, though – it's a more peculiar story. I posted a Help Wanted-ad on regarding the level compression in early 2013 (and indeed, later offered 100€ for someone who would create an editor) to which someone called Karethoth responded to. He made substantial work regarding the compression and created the wiki.

However, I got a message the 3rd of July 2013:

If you're still lost on that decompression problem, I can take a crack at it.  I'll need you to do the following:

1. Put a read breakpoint on the map data, just like you said you did in the want ad.
2. When you hit the breakpoint, hit "step out".  Write down the offset of the instruction you step out onto.
3. Reset the game and disable the first breakpoint.  Set execute breakpoints for the offset you wrote down and that offset - 2.
4. When you reach the breakpoint (even if it's before you expect it to happen), enable "trace once", "tabbed output", "squelch", and "cpu tracing".
5. Click run; you will reach the second breakpoint pretty much immediately.  Disable "cpu tracing"; there should be a text file near your rom called "name-of-rom0000000.log" or something like that.
6. Send me that file.  Any other information, e.g. notes on addresses and what they're used for, could be helpful, but if I need something I'll ask.
7. Wait.
8. ...
9. Profit!

The trick with really abstract disassembly (like compression or decompression routines) is to go through the routine and translate it instruction by instruction into a language you (and humans in general) understand better.  You don't necessarily have to finish & compile & emulate the routine (though it helps for error checking), you just come to a much better understanding of what's doing what as you translate it.  That's what I did when I was reversing decompression in Brandish 2, anyway, and apparently it worked out.

Regards,
Dave Hand
sqykly@gmail.com

To which I replied:

Thank you for your interest. Someone already dug into the routine and we've got a somewhat loose wiki under development. You can check it and probably contribute if you want to here.
We haven't done much progress. I guess the original supporter lost interest and I really can't help that much. :C
If you know a ambitious ROM hacker, I'm willing to pay 100 euros to anyone who develops a functional level editor for the game.

And our dialogue went on:

You may be in luck; I am a stupidly ambitious rom hacker drawn to seemingly lost causes, and I need money.  When you say "functional level editor", what are the qualifications?  Technically, you can edit whatever you want with a hex editor, but I'm guessing you'll want something more task specific.  Do you just need to be able to pick which tile goes where, then compress it and shove it in an IPS patch?  Or do you need other stuff like events, dialogue, some tools to see how far you can jump, etc?  Do you need to be able to load levels from the rom?  Do you need to be able to access (or at least not overwrite) existing levels from the new level in-game?

An I said:

Oh! Intriguing. And yes, there are a few key requirements for the editor.
-It must either have a separate/integrated level exporter/importer or a 'load everything on the fly'-kind of editor. Best case, both.
-A level editor, by definition must correctly display decompressed graphics, either straight from the ROM or from decompressed graphics files.
-It must have a physmap editor, a tile editor and sprite position/type editor as well as starting point and tide editor. This is a given.
-It must have a header editor for each individual level.
-The editor must be open-source.

Those are the absolute minimum requirements. Might be a tough job since to my knowledge, the wiki is incomplete and on a hiatus, so you might have to dig deeper to the compression routines yourself.
These are not required, but would be fantastic to have:
-It should be cross-platform
-It should have a credits editor
-It should have a condition editor for 'how many enemies must be killed until door opens' (if such conditions exist. I'm not sure if it's already defined in the level headers.)

Thank you for your interest. Of course, I understand if the requirements don't meet your standards but thanks so much either way!

To which he replied:

I went ahead and finished the work on the compression format; it's on the wiki.  It's actually quite similar to the compression I encountered on the Brandish 2 splash screen, though its use of intermittent callbacks like coroutines was an interesting twist.

I need to know how well the "physmap" data is understood and how much tile data is mapped out in order to decide how I'll prioritize this project.  I have a website (play5ong.com) that needs debugging work before I start collecting ad revenue from it, but if the tile data for this game, the header format, and the physmap don't need to be reversed from scratch, I can do this fast enough to not go broke and lose my server.  Otherwise, I really need to get 5ong running, and this will need to wait a bit.  Honestly, I love reversing engineering, tool-building, and the SNES, so I'd rather do this, but broke is broke =(.

Do you have a particular platform or language in mind?  I've been writing nothing but HTML5 & JS for a while, and it would really make a lot of UI stuff easy to make the level editor as a web or browser app, and that would necessarily be cross platform and open source.  It might be a little slower than a native code or java application, but it would work, and it could be done in a reasonable time frame.

I'm not going to post all the e-mails, but this is how it all started. We had now made a deal, and something incredibly ambitious, a functional level editor that works straight from the browser was under development. Not only that, but he also documented everything he found to the wiki – He even wrote a haiku for every enemy entry.

Needless to say, things looked prettty amazing. He sent me demonstrations about the user interface and we had decided upon a name, even (Kawabunga).



We were both sure we were nearing the release, and he sent me this the 27th of September 2013:

I just got done running some tests of the low-level modules in Node.js, and there's good news and bad news.  Good news:
- Compression & decompression work.
- At the very least, the Umihara-specific module and the IPS patch module work as browser code AND as Node modules, so if you want to make the editor a web service for whatever reason, you can put those (and probably almost everything else, but not yet tested) on the server side or the client side without any modifications.
- Both of those modules are free of egregious obvious syntax errors and crashes resulting from normal use.

The bad news is that I am going to have to do that re-allocation thing I was talking about, whether old levels need to be linked or not.  The issue is that the map data isn't contiguous in the ROM, and I have no idea what data is located in between maps, so overwriting it is fairly hazardous.  It isn't THAT big a deal, it's just another low-level thing to write and debug.  This is not at all the first time I've wished I had written the Brandish 2 editor re-allocator in JavaScript instead of Basic (actually, anything other than Basic would have been better).  So, anticipate a slight delay.

Unless of course you know something I don't about the ROM's layout, for instance one or more very large areas of totally unused space?

and soon after (emphasis mine),

Okay, re-allocation is done forever, and compression and decompression were both thoroughly tested by decompressing every map, recompressing it, decompressing it again to compare to the original decompressed version, then re-allocating all of them into the space they originally filled.  Back to the UI controller code for me.

Have you heard anything from Karethoth regarding his continued participation in the project?  I still have a couple of things it would be really cool to have tested, by someone who isn't me.

We were all super excited, knowing the release was upon us!

But... that was his last message.

I tried contacting him again and again and again, first because I was confused but later because I was worried something might've happened. I finally gave up hope in late September 2014.
To this day I still have no idea what happened. We were so close.



However, there's still hope. Someone called ellisong has started development on a tool named Riverback, and he/she seems somewhat active at GitHub. Truly, something to look forward to.

In closing, I want to inform that the level format used in the recently-released PC version is exactly the same as in the SNES/DS versions.

[KawaseFan]

My contributions include the unused enemies mentioned earlier, comprehensive Sprite rips, Map rips and the Atlas vid. I hereby give permission for Kawasefan.net to use and host all of the media.

Awesome, thank you!

Interesting to read those details about that attempt at a level editor.  It's really strange how the guy just disappeared like that after he'd made that much progress...

It'll be interesting to see how Riverback goes.

[Alc]

Hi Verneri!

I also made a ROM hack that links all levels together, removes the 30-minute rule and implements the unused enemies in some levels, but I made that as a /vr/-only novelty and I no longer have the link. I never even made a patch, I simply posted the .smc. Someone might have it though.

God damn, I need this ROM. Any pointers as to who might have it or where they might be found?

[Naulahauta]

I'm not sure if it's the latest version, but I sent you a PM. I can't guarantee that it's the one with unused enemies but I think this one's it.

[Alc]

Much appreciated.

Can you talk a little about how you did this:
https://www.youtube.com/watch?v=3Z9QRSCrj0I
it's a fascinating video to watch.

[Naulahauta]

Back when the video was made, I had recently finished the Umihara Kawase Map and Sprite rips. Around the same time I found Andy Dick's fantastic Atlas Vids. I knew I had to do the same for Umihara Kawase, so I asked for Andy Dick's help. The trick here was to render three separate passes of the TAS, straight from the emulator, as lossless, uncompressed videos. The first pass was the TAS but with the foreground layer visible only, then the same for sprites only and finally the HUD only. I would've done this myself but at the time I only owned a Mac and that particular BobWhoops' TAS works only on an obscure version of SNES9X for Windows.
Nevertheless, he sent me the movies and I laid the maps into my Adobe After Effects composition. After that, I put my movie on top of the composition and keyed out the background color.
The trick here is that Xkeeper had earlier helped me out with lua-scripting and had created a script that dumps given RAM addresses to a text file every single frame. I 'watched' the movie and got myself a massive text file with the RAM addresses of "screen X" and "screen Y" values on every row. I converted these hex values to decimal and added After Effects keyframe syntax to them, copied the text, and pasted it into the coordinate row of a Null object in After Effects. I then linked the movie file's coordinates to the Null Object's coordinates and tried to sync them as best as I could. The result was this:

As you can see, the Null Object's trajectory is pretty obvious, moving a couple of pixels every frame, and the 256x224 SNES screen is locked into it from the top left corner. When she moves one pixel to the right, the field moves a pixel to the left, and the Null Object moves a pixel to the right. It's simple really.

This video probably makes it even more obvious, as that's the first demo I showed anyone (and I hadn't removed the HUD elements yet)

However, I had to manually add sprites like the moving platforms and the tadpole. That was painful. The original plan was also to extend the levels to full HD (like I did in the last field) but I lost interest.

[Commando125]

Hey. I'm amazed my Riverback project was even found. I've been playing the Umihara Kawase games for a couple months now and a friend on some site I go to wanted me to make a level editor for the game. I've been working on figuring out the SNES game for a couple of weeks while I worked on a portfolio project at the same time. I do need some help on knowing where to go next.

I know how to decompress the level data. Some rough notes on the decompression function is posted below, which probably won't make sense to anyone except the person who wrote it (me). The decompress function was uploaded just now to my small project. I was also able to decompress some graphics with that function (some are at 0x70000, or $8E8000), it was basically the "object" tiles for the Umihara Kawase fields.

I found some information on that wiki that I have been struggling to find; and I might have notes that are not on that wiki. I'm going to try to get my notes kept together and accurate with the one from the wiki. Is the wiki active at all, or was it active until 2013?

Let me know if something is needed. I'm excited others are working on figuring this out too.

EDIT:
  I checked out the wiki. He knows more than I do currently. Wish we could get in contact with him. I never wrote the compression algorithm yet, just the decompression one. There are two places at 0x70000 and 0x68000 that have compressed graphics, and the decompression algorithm I wrote can decompress them into a format viewable by a ROM graphics editor. I was able to insert uncompressed levels into the game though by expanding the ROM, inserting "00" every 9 bytes, and changing the level header pointer. More levels can possibly be inserted to if space is found within the first bank of the ROM for the level header pointers.

  I was thinking that someone else should start the level editing project, because python and Tk might not be the best tool for this. We probably have enough data to edit the levels, but maybe not the enemies/objects. Writing the compression algorithm shouldn't be too difficult. I would love to help with the project though if someone takes over.

@Naulahauta: Do you know how to draw graphics from the ROM? I don't know how.

Code Select Expand


4096 tiles per field
Tiles are made up of 2 bytes: a 1 byte tile type and a 1 byte tile property
tile types are the tile # in the bank

The algorithm reads bytes 1 at a time.
Every read of a POS byte sets a counter equal to 8.
This counter allows to read 8 non-POS bytes before having to read another one.
The POS byte tells where the position(s) of a LZ byte(s) is/are via a bit set.
If a non-POS byte is read, the counter is decremented by 1.
EXCEPTION: if a LZ byte that is read has 0 for the left 4 bits, the counter is not decremented, and is only decremented by 1 after the next byte is read
The leftmost bit is byte 1, the rightmost bit is byte 8.
To end decompression, read #$00 four times when a POS bit is set

The LZ bytes copy data from behind it.
LZ byte: TTTTLLLL
T: total amount of bytes to place (+1)
0bTTTT + 1
If TTTT is 0, the byte after the LZ byte is how many bytes to place.
L: amount of unique bytes to copy from behind the LZ byte.
0b10000 - 0b0LLLL


[Naulahauta]

Hello and welcome. I found your project by doing a simple "Kawase" query on GitHub.

The wiki is not active any more, but for the sake of preserving the knowledge it's still up. It would probably be wise to mirror it to Data Crystal or at least make a local backup.

If you read the posts earlier, you probably already know that the man behind all the info can't be reached. However, I just noticed that he's last seen on Stack Overflow a while ago so you might be able to get in touch with him there. I'm glad he's not dead.

I am not an expert on ROM hacking, but I know for certain that ROM images can be expanded with a tool called Lunar Expand. I don't know if it helps in this project, though, but if one could repoint the levels to the expanded areas, they would likely fit.

To be fair, the destiny is in your hands. Sqykly's gone full hermit, Karethoth's gone, and I can't program*.

What do you mean with "draw graphics from the ROM?"
I'm pretty sure I know how the sprites work – you might want to read my case study (it's from the DS version but the same formats might work in the SNES version)


In closing, I can't comment on whether python and Tk are the "best tools for this" but if there's an advantage, it's cross-platform at least.

*) I have gone through the Codecademy Python course, but that's it.

[Alc]

It's simple really.

It's anything but simple. Thanks for the thorough explanation, though, and for all the hard work. I would never have thought to do it that way.

Great to hear from you Commando - hope you keep working on it.

[Commando125]

After a few days of troubleshooting, I was able to decompress every graphic for the banks of the level stages/fields. Since I saw some interest on this forum for ripping media resources from Umihara Kawase games, the graphic banks are attached below  . Two banks per field. The first 15 tiles in the bank (the ones with the same double digit) are 15 palettes for that bank. You will need a tile viewer (like Tile Molester) and the 7zip program to open them. Also note that there are some graphics that are uncompressed in the ROM (such as sprites and backgrounds).

The compression algorithm isn't done yet, but it can compress to something the game can read. It just won't compress the same way as it originally was before.

[Naulahauta]

Are you sure everything's okay? I downloaded the archive but it always decompresses files with the size of 0 bytes.

[Alc]

Download worked for me (160KB) and extracts ok.

[Commando125]

Progress is being made, but it can only display tiles (no objects) and I'm missing two hard-coded palettes for the tiles to select from, but I don't think they are ever used in the game for tiles. Color 0 for the palettes should probably be transparency. I'm wondering what the layout of the editor should look like. The level data and graphics are all read from the ROM itself.

EDIT: I fixed the transparency, but it isn't in these screenshots

[Naulahauta]

Looks absolutely amazing!! Love the UI's elegance too, looks really smooth and after all, all of the levels are the same dimensions.

[Alc]

So exciting to see this. Great work!

[Naulahauta]

I take it there are no pre-defined palettes or formations for any tiles, and one would have to always set the "right one" to the tiles in question?
I mean, the walkable blocks look reasonable as gray or blue or light green, but the trees have to always be the tree-palette and look obviously broken if used with any other palette.

Instead of displaying the 8x8 tiles as a single-palette porridge from $000 to $1FF, would it be a good idea to include a "typical tile cluster definitions" file (an xml or something) that would include pre-built objects with their respective, real palettes? I wouldn't want to build a rock or tree formations from 8x8 blocks one by one when doing the level.

[Commando125]

Since the tiles that can be used can be easily changed per level, I can't see pre-defining palettes or formations happening. The tiles would be in a different position for every bank on every stage.

I am rewriting/porting the editor from Python to C# .NET with WPF. I think the GUI will be a lot better, along with a possible performance increase, because Python's UI options are terrible (you have to convert an image to another image to be able to display it on the UI).

[Naulahauta]

Since the tiles that can be used can be easily changed per level, I can't see pre-defining palettes or formations happening. The tiles would be in a different position for every bank on every stage.

I see, sounds reasonable.

I am rewriting/porting the editor from Python to C# .NET with WPF. I think the GUI will be a lot better, along with a possible performance increase, because Python's UI options are terrible (you have to convert an image to another image to be able to display it on the UI).

While C#/.NET/WPF is inherently Windows-only, many such programs can be run on UNIX/Mac/Linux with Mono, but you probably knew this already. What I'm urging is that please refrain from using any Windows-only DLLs or obscure API calls if you can, for cross-platform compliance.

Looking forward to this!

[Commando125]

I would have to switch from WPF to support it on other operating systems, but it doesn't seem too bad to switch to Windows Forms or GTK#.

[Naulahauta]

I would have to switch from WPF to support it on other operating systems, but it doesn't seem too bad to switch to Windows Forms or GTK#.

Gah, now that I looked at the WPF page I see you're correct, it's not supported.
I don't want to stall your project or put any unnecessary pressure just because a handful of users might benefit from the switch from WPF to GTK, so do as you see fit.

On topic - do you have any GUI plans or wireframes?
Also what about the physical attributes of the tiles? Does one have to first draw the level and then draw the physmap?

[Commando125]

Not sure what my GUI plan is or what the physmap editor will be like. I converted the program to C# with WinForms though.

[Naulahauta]

Looks pretty nice! Can levels be exported/imported?

Also – someone on /vr/ said they'd be happy to contribute to the GitHub project and maybe even port it.

[Commando125]

Not yet. It shouldn't be hard to do though. I'm going to make a TODO list on what to do on the editor. I will also need more information on the objects/enemies; seems to be lacking. Will keep you updated.



EDIT: Added a tile selector. Doesn't modify level yet.

[Naulahauta]

What's the tile selector used for?
I don't think selecting a single tile for "painting" the level with is that fun because the tiles are so small. If you've tried out Lunar Magic, its multi-tile selector (select area with left mouse button, paste with the right one) is very intuitive to use, but then again these are two very different games.

[Commando125]

Super Mario World has "objects" with a length and width. Umihara Kawase doesn't have that, its two bytes per 8 pixel by 8 pixel tile. Can you tell me a bit about the multi-tile selector? Might be possible. I can probably zoom in the tile selector and level editor by 2x to fix the small tiles, but I would have to add scroll wheels on the right and bottom sides.

[Naulahauta]

I know, but SMW also has "map16 blocks" which are essentially 16x16 equivalents for SMW to what these 8x8 tiles are in Umihara Kawase.

I made a quick and dirty demo in After Effects to show what I mean.
VIDEO LINK

The animation seems a bit fast, now that I look at it.. but ehh, you probably get the point.

• Dragging the mouse on-level shows instant preview of what it would look like if placed.
• Air tiles act as "transparent" (maybe this could be changed on context?)

Left click: select tile
Left hold: select area
Right click: paste tile/tile cluster
Right hold: "paint" with tile or tile cluster
Wheel click: de-select

This is how Lunar Magic handles its map16 tiles iirc, and it's a very robust method of making large complex shapes from smaller pieces.

[Commando125]

That was a well-made demo. After I get some free time, I will see what I can do.

[Commando125]

I got level tile pasting to work in-game, but no "map16 blocks" feature yet. Adding the map16 feature might require a small rewrite and a bit of work. Tried to reorganize the GUI a bit, but I might zoom the tileset by 2x and reorganize it back to how it looked originally (I keep clicking level by mistake instead of palette), but with the tileset zoomed in.

Next month, I might have less time on this project due to professional work.

[KawaseFan]

Nice work, much appreciated that you've spent your time on this.

[Naulahauta]

I'm with KawaseFan. It's fantastic to see such sporadic progress and the fact that you went the GitHub route makes the project even more appealing.

Good job! Do you have Paypal or something?

[badlose]

Awesome work! CARRY ON!

[mfgreth]

I've been away for so long, and I come back to all this. This thread is magic. You are all magic.

[Commando125]

I was able to add multi-tile selecting for the level tiles and tile painting for tileset tiles. I'm having issues with highlighting the selected tiles, but it seems to work non-visually.

I'm also looking for ideas on what the physmap editor should look like. I'm open to suggestions.

[matt]

Awesome work so far, consider me excited and subscribed!

[Commando125]

Level tile highlighting. I want to start work on the physmap editor now. I'm sure of what bits represent the physical properties of each tile, so I will have to research it.

[Naulahauta]

How many different physmap attributes are there?
I can make a mockup if you can confirm the exact amount (and type) of physical tile attributes.

Right now I can only think of 16 types of walkable slopes, 1 type of solid, 2(?) types of bird landing spots, 1 type of  spawn-safe zone and 1 type of spikes and 34 different conveyor setups (16 clockwise slopes, 16 counter-clockwise slopes and solids for both directions)

[Commando125]

Here is what I have so far using my editor to read the physical properties of a selected tile. I'm starting to think that the physmap tiles can also be used to locate where doors are (there is a number 1 where you open the door), where ladders are (there is a number 1 where you can grip the ladder), where spikes are (by checking if the tile above a solid tile is a spike tile), and where objects and enemies are according to their type in the level header (I have seen tiles numbered 2 to 7 where umihara and the objects/enemies are, including a number 1 tile alongside the right of them (to determine facing?)). Use this editor link if you want to find out what tiles are what: https://www.dropbox.com/s/yavaz3uehvf0j5h/riverback_physprop.zip?dl=0.  Make a backup of your rom before using, since it auto-expands the rom on save (to fit more data). It cannot edit physmaps yet, but it can edit tilemaps and it will help display what a tile's physmap value is. If you want to edit phystiles in emulator RAM, look in $7E4000 - $7E51FF (there are four rows of tiles above and below the level). Expect bugs. Also, do not open a rom that isn't an Umihara Kawase headerless rom. If the program throws an exception, you have the wrong rom.

PHYS TILE PROPERTIES:
    $00: 0000 0000: Air
    $01: 0000 0001: Ladder (Left edge), Door?
    $02 - $07: Object placement? From enemy types in level header?
    $39: 0011 1001: Spike?
    $3A: 0011 1010: Spike?
    $40: 0100 0000: Solid
    $41: 0100 0001: Spike? Ice?
    $42: 0100 0010: Spike?
    $45: 0100 0101: Escalator left
    $47: 0100 0111: Escalator right
    $80: 1000 0000: Solid slope bottom-right: low
    $81: 1000 0001: Ice slope bottom-right: low
    $87: 1000 0111: Escalator slope bottom-right: low
    $88: 1000 1000: Solid slope bottom-right: high
    $89: 1000 1001: Ice slope bottom-right: high
    $8F: 1000 1111: Escalator slope bottom-right: high
    $90: 1001 0000: Solid slope bottom-left: low
    $91: 1001 0001: Ice slope bottom-left: low
    $98: 1001 1000: Solid slope bottom-left: high
    $99: 1001 1001: Ice slope bottom-left: high
    $A0: 1010 0000: Solid slope top-right: low
    $A8: 1010 1000: Solid slope top-right: high
    $B0: 1010 0000: Solid slope top-left: low
    $B7: 1011 0111: Escalator slope top-left: low
    $B8: 1011 1000: Solid slope top-left: high
    $BF: 1011 1111: Escalator slope top-left: high
   
    SRVH hEeI
   
    S: slope (solid if s = 1 , even if S = 0)
    R: reacts to environment? (tilemap tiles above it, or on it?) (ladder, door, spike, ice?)
    V: v-flip for slopes
    H: h-flip for slopes
    h: slope height (0 = low, 1 = high)
    E: escalator (solid if E = 1, even if S = 0)
    e: escalator direction (0 = left, 1 = right)
    I: ice? spike?

[Naulahauta]

Very interesting.
I ripped ZST Save States of each level back in the day, and because ZSTs are simply a frozen piece of what's loaded to the RAM at the moment, the physmap is also there. I compared these ZST-physmaps to the physmaps that your editor shows and there's some strangeness:

The physmaps in the ZSTs are 72 rows, while your editor shows only 64 rows. Which one is correct?

ˆ I pasted the ZST's bytes over the level map. It's almost exactly like your editor shows the bytes but, for example, your ladders are "01 00" while the ZST's ones are "01 03". Also, the trajectory of the rising platforms is a bunch of "03 03 03 03" 's in the ZST, but "07 00 03 07" and "06 00 03 07" in your editor.

Because can't trust my ZSTs, I can't just copy every byte in the physmap, paste them over the level in Photoshop and write the different Phys Props down whenever I see a new one. Although your editor displays them probably the right way, it would take forever to click on a tile, write its phys prop down, click on the next one, etc.

But anyway, once we know for certain exactly what each byte does, this should be easy. I just need a reliable way to get the raw byte dumps of the physmaps so I can make direct comparisons and figure out what tiles are what.

As for the User Interface, here's my plan. VIDEO LINK. Again it's a bit shoddy vid at parts but you probably get the point – the physmap editor would work exactly the way the field editor does, but with its own separate annotated pictogram set with physical tile representations from $00 to $FF. In the video, it's obviously unfinished, but filling them all should prove trivial if they're bitwise anyway.
One could also be able to simultaneously edit or view the physmap and the field, or just the other.
I'm not sure how to implement it but I thought maybe adding a raw access method to the bytes could be useful too, but it's shown only briefly.
I also added a grid and a new field, the scratchpad, where the user could construct their own tile/phys arrangements for easy access and without polluting the level itself.

A couple of other suggestions, some probably already a work in progress but anyway:
• Add hotkeys to number keys 1 and 2 for previous palette and next palette.
• Add hotkeys to up/down and left/right for tile (and tile cluster too) flipping accordingly.
• Now that the air tiles are 'transparent', erasing large structures becomes a chore because I can't simply copy a cluster of 'air' and paint over stuff with it
• Add undo

Lastly, I'm sure these are probably obvious bugs/not features yet, but just to make sure these aren't Mono-specific issues:
• Nothing updates until I let go of my right/left button. When I drag the area I wish to select, I see the light box over the area only when I let go of the left button.
• When I try to paint over the level with a tile cluster selected, it only lays down it once even if I hold the right button. Continuous paint with a single tile works well though.
• When I've selected the cluster, moving around doesn't give a preview of how it will look, so I have to commit the cluster to see how it looks in all cases.
• I got a crash when I entered things I shouldn't have entered to the "Palette" field.


Otherwise a solid job – really really looking forward to this.

EDIT: wow that wasn't very coherent at all. I think I might need some sleep, but I'll get back on this later. Happy new year!

[Commando125]

The physmaps in the ZSTs are 72 rows, while your editor shows only 64 rows. Which one is correct?

The 64 rows is correct. The game adds 4 hardcoded rows on the top and bottom that mirror the top and bottom edges of the level respectively.

It's almost exactly like your editor shows the bytes but, for example, your ladders are "01 00" while the ZST's ones are "01 03". Also, the trajectory of the rising platforms is a bunch of "03 03 03 03" 's in the ZST, but "07 00 03 07" and "06 00 03 07" in your editor.

I am confident that the physmap tile values shown are correct. I have seen the game modify the tilemap tiles after the data was decompressed to RAM. It might be modifying the physmap tiles too.

I just need a reliable way to get the raw byte dumps of the physmaps so I can make direct comparisons and figure out what tiles are what.

For one reliable way, set a write breakpoint at $7E5200 and copy the tiles from $7E4000 - $7E51FF in RAM to a new file via hex editing. If you need level exporting right now, it shouldn't be much trouble to put in.

As for the User Interface, here's my plan. VIDEO LINK. Again it's a bit shoddy vid at parts but you probably get the point – the physmap editor would work exactly the way the field editor does, but with its own separate annotated pictogram set with physical tile representations from $00 to $FF.

I'm liking that interface. Well done on that video. It presents your ideas well.

I'm not sure how to implement it but I thought maybe adding a raw access method to the bytes could be useful too, but it's shown only briefly.

For tilemap editing, I see no point in raw byte access. All the tiles that can possibly be selected are in the tileset canvas. For physmap editing, I can see a use for raw byte editing, but the physmap tileset idea you had would make that use not very useful since I could select any tile I wanted from there.

I also added a grid and a new field, the scratchpad, where the user could construct their own tile/phys arrangements for easy access and without polluting the level itself.

The grid should be no trouble to add. I really want a scratchpad too, but implementing a scratch pad would take some work. I would have to treat the scratchpad, in the way I'm currently programming this, as a second level to paste tiles on. Also, the scratchpad would have to be cleared each level because each level has a different tileset arrangement.

Add hotkeys

I can do that with no problems.

Now that the air tiles are 'transparent', erasing large structures becomes a chore because I can't simply copy a cluster of 'air' and paint over stuff with it

I can either make air copyable or un-copyable. Each choice would have different strengths and weaknesses. Un-copyable air can help place structures in the level that have air around it, such as trees and ramps, but any air that you actually want there you would have to draw over by selecting the air tile in the tileset panel. Copyable air can make erasing easy, but makes it harder to place structures /w air around it because then you have to draw on the air that you don't want, which would take more work than having it transparent. I can make this option toggleable if you want.

Add undo

This is nice to have, since pasting a tile cluster that you don't want over your work can be devastating, but requires more work than you think since you have to think of every case that the user can undo from and how to store past actions from the user as well as the data from before. It isn't too high on my priority list yet.

Nothing updates until I let go of my right/left button. When I drag the area I wish to select, I see the light box over the area only when I let go of the left button.

This was intended because I am currently unsure of how to highlight tiles while the mouse cursor is being moved.

When I try to paint over the level with a tile cluster selected, it only lays down it once even if I hold the right button.

I thought about allowing this, but I can't see many use cases for it except for copying tile clusters with one tile value faster. It is too easy to accidentally drag your mouse when pasting with a cluster and then screw up your work.

When I've selected the cluster, moving around doesn't give a preview of how it will look, so I have to commit the cluster to see how it looks in all cases.

I am currently unsure of how to draw tile clusters on the level while the mouse cursor is being moved. Right now, the level editor can only redraw tiles that it has stored in the level itself. Having to draw tile clusters without modifying the level can cause some redrawing issues such as tiles being missed to redraw.

I got a crash when I entered things I shouldn't have entered to the "Palette" field.

The accepted palette value inputs are probably out of range. I will look into that.

Thanks for your input.  It gives me ideas on what to do with the editor. Let me know if you need something or if you find valuable information.

[Naulahauta]

Thanks for the thorough replies. What I forgot to show in the video is that if you've enabled simulatenous physmap and tilemap editing and wish to work with tiles instead of tile clusters, you would have to pick both, the tile from the tileset and a pictogram from the physset. Selecting a new tile would still keep the phys prop selection, and vice-versa.

I'll look into the physmaps when I have some time. Using a breakpoint-enabled emulator is a chore because I'm on a Mac, but if I virtualise/use WINE I can probably make Geiger's SNES9X debugger work.

For the Undos, you said:

every case that the user can undo from and how to store past actions from the user as well as the data from before

Maybe I'm not thinking this through but in my opinion the only cases where the user can (or needs to) undo from are accidental tile (or tile cluster) painting operations.
Maybe the workflow could be so that every time you paint a tile or a cluster, a history state is added to an invisible "undo stack" xml file (or in the computer's RAM I guess) that stores a {old/new/how} message per row.

• If you select a 3×2 cluster from any point in the level, starting from the top left corner, that contains the tiles 00 03, 00 03, 10 03, 10 03, 00 03, 01 03 with the physprops 00, 00, 01, 01, 00, 00 and paint it over the level's coordinates x:8 y:14, the program would, before overwriting the section, store the original tiles you're destroying as variable old. Because your selection was 3×2, it would know to read three columns and two rows, from left to right (because the selection was made from the top left). If the original tiles were 20 03, 20 03, 20 03, 20 03, 20 03, 20 03 with the physprops 81, 81, 81, 40, 40, 40, the old variable would be 200320032003200320032003-818181404040. And as you click, the changes are made, the tile cluster is painted over, and the new variable is added as the undo stack's current row's other pair. The stack would have one row now, which is:
old:200320032003200320032003-818181404040\new:000300031003100300030103-000001010000\8-14-topleft-3x2

If the user were to push Undo after that, the program would paste the latest row's old bytes as a 3×2 cluster, to coordinates x:8 y:14, and row would be discarded (not deleted because Redo is cool too).
For single tile painting operations the format could be different, with simply the first variable being "what tile with what physprop" and after that, a stream of "x, y, what tile and physprop used to be here" value pairs.

Note: I am not a programmer so I don't know if this is efficient or wise or makes sense. It's just what I envisioned.

[Commando125]

What I forgot to show in the video is that if you've enabled simulatenous physmap and tilemap editing and wish to work with tiles instead of tile clusters, you would have to pick both, the tile from the tileset and a pictogram from the physset. Selecting a new tile would still keep the phys prop selection, and vice-versa.

I can make an undo for the last tile cluster pasted. Having undo for just single-tile pasting wouldn't be as painful to fix.

Using a breakpoint-enabled emulator is a chore because I'm on a Mac, but if I virtualise/use WINE I can probably make Geiger's SNES9X debugger work.

Try this one out on WINE: https://www.dropbox.com/s/2p7xtnpwpqhp0xx/bsnes-plus-r161.zip?dl=0. That is the emulator I used to reverse engineer the compression/decompression algorithm, AND it allows you to edit the RAM in-game.

I added a quick function to Riverback to save (and write the physmap to level) the physmap in the file menu: https://www.dropbox.com/s/yavaz3uehvf0j5h/riverback_physprop.zip?dl=0. Use it to help you find out what those physmap tiles mean.

[Naulahauta]

Thanks, that editor is really helpful. I've done some research and here's some conclusions:

////////
Physmap bytes:
• 00: Air

• 01**: always a word where the first byte implies that an action will happen when the player pushes up and the second byte implies what the action is:
Where ** is:
00: climb a ladder
01: enter first door
02: enter second door
03: enter third door
04: enter fourth door
05-FF: the game will crash
NOTE: There can also be only one door instance (0101 or 0102 etc.) per level or the game will crash.

• 02** and 03**: always a word where the first byte implies direction and the second byte implies sprite number (up until 06).
02: sprite is facing left
03: sprite is facing right
Where ** is:
00: Player starting position (There can only be one. If there's two instances of 0200 or 0300, the game will crash)
01: sprite 1
02: sprite 2
03: sprite 3
04: sprite 4
05: sprite 5
06: sprite 6

• 03 07: Spawn-safe area. Enemies killed cannot respawn here. (special case, has nothing to do with sprites or facing left or right)

04: ?

05: If the field's sprite 05 is a bird, they can land in this spot.
06: If the field's sprite 06 is a bird, they can land in this spot.
It would be wise to remember that if you use birds in your levels, set them to slots 05 or 06. Setting them to 02 and 03 will cause a conflict and the game will crash.
////////

What bothers me, though, is the fact that in some levels 41 is a spiky solid, but in some levels it's a slippery solid. It appears that there can't be one-size-fits-all pictogram set, because the meaning of the physprops changes between levels.
It seems that conveyors and glass are exclusive.. you can't have both in the same level, at least by normal means.

There are the 14 still unknown bytes in the level headers. I looked at them and they're definitely related. Fields 24, 35, 42, 33, 23 and 28 are the only ones that have glass surfaces on them. Here's their 14 unknowns:
field 24: 3C B4 5A 78 00 00 00 00 2A 00 00 0B 30 10
field 35: 3C B4 00 00 00 00 00 00 2A 00 00 00 00 10
field 42: 00 00 00 00 00 00 00 00 2A 00 00 00 00 00
field 33: 3C C8 50 8C 00 00 00 00 2A 00 00 00 00 00
field 23: 3C 78 78 3C 00 00 00 00 2A 00 00 00 00 00
field 28: 00 00 50 B4 00 00 00 00 2A 29 2B 00 00 33

These 14-byte arrays are also the only ones that contain 2A at that specific spot. I turned the 2A to 00 on field 23 and that successfully disabled the slippery surfaces and made them grabbable.

It appears that some features like spikes, conveyors, moving platforms(?) and the hanging bridges(?) must be enabled per each level by doing something to these bytes.

To reiterate, here's the unknowns of all levels with conveyor belts (emphasis added):
field 00: 3C 64 28 64 00 00 00 00 00 00 00 00 00 00
field 01: 3C 64 00 00 00 00 00 00 00 00 00 00 06 05
field 18: 3C DC 00 00 50 B4 00 00 29 00 00 00 07 05
field 43: 3C 64 28 64 00 00 00 00 00 00 00 00 00 04
field 49: 00 00 00 00 00 00 00 00 00 00 00 28 04 05

It could be that 00 sets up conveyors, maybe. Field 18 is different because it's the only level in the game that has conveyors and spikes.

[Commando125]

Great work on those finds! That really helps us with figuring this out. I was wondering if you can think of what tile should be what graphic for the physmaps? Or should they just be displayed as bytes?

[Naulahauta]

I started drawing the pictograms just today, if that's what you mean. I haven't finished them yet but there's two sets, a typical 8x8 set and a higher resolution one in case a zoom function gets implemented.

[Commando125]

Can you draw the 8x8 one at 1x zoom and rid the grid lines? I can add the grid lines at runtime.

[Naulahauta]

Sure thing. Here you go.


It's still missing many pieces and I'm not sure how to represent the 01** 02** 03** parts though. I'll figure something out.

[Commando125]

Thanks. That helps a lot. I never implemented drawing from a tile graphic that's from a .PNG tileset yet. Only from SNES graphic data. It will take a while to add that in. I will start working on it after I search for a short-term lease apartment. If you want byte tiles displayed, you might want to add that too, since there are tiles missing. Make sure the font is legible. Always make byte 0 clear, not 00 like in the video. I think it's easier to see without 00 everywhere. If you don't want to do the byte tiles, I can take a shot at it.

P.S. You probably want to go with a 3x7 font for each digit if you do the byte tiles.

[Naulahauta]

3x6 will give some more space. Here you go

[Commando125]

It looks like you are doing the physmap tileset in vector graphics. Make sure to save the vector file. I decided that I need a 1x (8x8) version and a 2x (16x16) version. I can probably scale the 2x version down to 1x in the editor though.

[Naulahauta]

That is true, but the fine detail is lost if I simply scale down the vector version to 8x8. It is easier for me to work in a vector environment with 8x8 tiles, and scale those tiles up to 16x16 and fix some details.
I'm not home right now but I'll send the stuff over as soon as I can. However, here's a 16x16 set of the bytes.

[Commando125]

Hey. I reformatted some code around to make life easier later. I'm going to look at some apartments today, but I definitely want to work on the editor later. Work on the pictograms if you can. I would represent 1-7 as just a large bold number without the 0 digit. (make it stand out maybe from other byte numbers?).

Do you have a specific order in which you want the editor features done?

[Naulahauta]

While you're right, the fact that you must always have a byte from 00 to 04 after 01, a byte from 00 to 06 after 02 or 03 (except in the special case of 0307) and that there can only be one 0200 or 0300 per level has to be emphasised strongly. This, and the fact that some unknown byte combinations can easily crash the game. This is the best I came up with:


While not a tileset, it's a word-set. You read two-byte columns. The first 16x8 column is empty because 00**s aren't special. The second column has representations for ladders (0100) and door exits (0101, 0102, 0103 and 0104).
The third column represents the 02** words like the starting position and the sprite numbers and their initial directions, and the fourth column, the 03**s is identical but with the added special spawn-safe pictogram for 0307.

Whenever the editor detects that there's a tile 01 02 03, it checks the byte next to it and uses this other tileset accordingly.

I wouldn't also mind a status bar/messages/console to the bottom of the editor that has warnings like "Multiple starting points detected. Expect crash" or "01FB detected at line 13. Expect crash" or something.

Do you have a specific order in which you want the editor features done?

Not really. I don't really know what's the easiest for you to implement first and I want you to be comfortable with them.

Here's the standard pictogram set with the added 01-07 as well.



Hope you find a nice apartment!

[Commando125]

I like that word-set idea. Would definitely help in telling what is what in the physmap editor. I'll add a status bar at the bottom, but I don't know what messages it will display yet.

[Naulahauta]

That's great. Also, now that I think of it, maybe the program could even prevent you from saving if it detects two starting points (evidently a guaranteed crash)?

One last thing: for debugging purposes could you add a "export all tilemaps" and "export all physmaps" function?

[Commando125]

Here is something that you can play around with. No physmap editing/display yet. It has level export and import, grid display, and the byte checkbox does something. I did not fully test export/import, but it seems to work when I imported an exported level 4 to level 1.

https://www.dropbox.com/s/ych8gdwesv9o8xd/Riverback_LevelExportTest.zip?dl=0

For exported level offsets:
0x00: Checksum
0x0B: Header
0x30: Physmap
0x1030: Tilemap
0x3030 - 0x3031: Tile index amount
0x3032: Tile indexes
0x3132: Last 6 bytes of palette indexes

[Commando125]

Does this seem correct so far? I can always change how it displays. I also need the wordtiles updated with anything you can find out. Please number the word bytes too (00 01, 00 02, etc.), even if it doesn't represent an object.



Edit: Made a mistake. Reuploaded new image.

[Commando125]

I made a test version that allows drawing and placing phys tiles, but there are minor graphic errors in the grid. I was able to draw a ladder and the phys props for the ladder and use it in game. I also moved a door somewhere else and it worked. There are no warnings yet when saving the ROM with bad phystiles or when opening a ROM that isn't a Umihara Kawase rom. I blame my friends playing Dragons Dogma for making me want to work on this a lot today, because I can't even run that game and they never shut up about it.

https://www.dropbox.com/s/uqs1vlto48yvz91/Riverback_phystiles_test.zip?dl=0

[Naulahauta]

Thanks! I updated the wordmap and the physmap.
I still haven't figured out why some levels substitute some phystiles for others, but here's a list of conflicts (I'm talking about the numbers shown in your editor, not actual field numbers):
• Clockwise conveyors sometimes mean "Boss will start attacking you" -tiles (Level 63 and 61). A mystery in level 4.
• Ice solids sometimes mean spikes (Level 12, 15 and 20, many others)
• Ice and spikes can still coexist, like in Level 52







The new User Interface is really sleek now! Good job.

[Commando125]

I was thinking. How are people going to figure out what makes up a word tile?

I have seen some weird things with those phystiles, such as 5 for bird spots instead of 6 and spike tiles having different values in each level. I think I'm going to make the header and tile index editors before moving onto word tiles. Maybe we can figure out what the inconsistencies mean with the header displayed per level. They are needed more than word tile displays.

[Naulahauta]

A well-written readme will do. If that's not enough, I'm willing to do an annotated video tutorial on how to use the editor.

A 05 alone is always a bird landing spot if the level's sprite slot #05 is a bird.
A 06 alone is always a bird landing spot if the level's sprite slot #06 is a bird.

A 04 alone can mean a platform trajectory, but birds have to be assigned to slot #05 or #06.
A 05 alone can mean a platform trajectory, but birds have to be assigned to slot #06 (maybe #04 too).
A 06 alone can mean a platform trajectory, but birds have to be assigned to slot #05.
A 07 alone can mean a platform trajectory, but birds have to be assigned to slot #05.
Bytes 04, 05, 06, and 07 can be used all in the same level for platform trajectory assignments, but that means no birds or little fishes.
Furthermore, bytes 04, 05, 06, or 07 define the "X" sprite that appears on top of the boss door.

[Commando125]

Interesting finds. Here is something I am working on. It allows you to select what field tiles to use in the tileset. There are a couple levels that have less than 15 tiles remaining, and I originally thought the max selected tile index limit is 497 (0x1F1). I'm curious on how a couple levels (Level 22, Level 46 (the upside down field 1)) can get away with more than 0x1F1 tiles selected. I'm also wondering why byte 8 can be a bridge platform or it can be the moving small platforms into the spikes in field 15 (and other fields).

[Commando125]

The tile index editor is finished. I shifted some tile indices around and the results look like the ones in the editor. I am going to move on to the header editor.

[Commando125]

This version will allow you to edit parts of the level header. Let me know if you find anything out; I'm probably going to take a short break. Expect bugs.

https://www.dropbox.com/s/45e3d7s6uv57p6t/Riverback_headertest2.zip?dl=0

[Commando125]

I'm going to review the unknown header data for each level. Maybe I can figure something out. I'll just post findings in this post until you reply.



Findings:

Header bytes $14-$1A seem to be object types for tiles $01-$07. It seems to affect tiles $40-$47 for moving/death/ice, maybe $39-3F.

There is an inconsistency in that wiki you linked me, where $1C is water height. Water height is actually 2 bytes, from $1B to $1C. $1C is always set to 1, because if it is 0, the water takes up half the level height and disappears when you are below the wave line.

Tile 45 seems like a faster escalator than tile 47.

[Naulahauta]

Header bytes $14-$1A seem to be object types for tiles $01-$07. It seems to affect tiles $40-$47 for moving/death/ice, maybe $39-3F.

DUDE

DUDE

This might be the final piece of the puzzle! It seems so obvious now.

The next few weeks are going to be somewhat busy for me but I'll try to squeeze in some time to look into these bytes too.

If we can figure out what bytes correspond to what kind of tiles, there isn't really anything stopping us from doing something that one would consider "a full hack"

A few things I've forgot to mention/ask, though:
• Do you know where the demo level data is stored and where the demo movement comes from?
• How trivial would it be to implement a title screen/game over/credits editor? I know the title screen/game over format but I don't know where the credits are...
• I have the "Starting amount of lives" and "30 minute rule on/off" -offsets written down somewhere. They would be a nice addition too, I'll let you know asap.

[Commando125]

I think the demo level is an actual level, but I'm not sure where the demo movement is.
The title screen and game over backgrounds are easy to edit and I think they are not compressed. Not sure about the giant game over fish.
Editing the credits would require finding the text of the people in the credits and changing it from there.
Starting lives are stored in $7E025A. Not sure what the 30 minute rule is.

Tiles $0C - $13 seem like 16-bit words for something. When I change it before loading a level, it seems to do nothing. Definitely seems different from the other 7 bytes.

I'm going to be starting work next week, so I might have less time, but I should get free time when at home. Finding out what each object byte is will save some work, but explaining what each one is in the editor might be an issue. If you edit the header, make sure to click "Apply Changes" before you save the level.

https://www.dropbox.com/s/b1rw7j2iwdpcrfb/Riverback_headertest3.zip?dl=0

[Commando125]

Theses are all the object values I seen in all the level headers, all in hexadecimal. The other values you shouldn't need to worry about because they are unused.

0x00, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0B, 0x0C, 0x0E, 0x0F, 0x10, 0x11, 0x18, 0x19, 0x20, 0x21, 0x22, 0x26, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, 0x2E, 0x2F, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B

Those 2-byte words in the header I talked about:
0x0000, 0x14FF, 0x1E96, 0x1EA0, 0x1EB4, 0x2864, 0x2896, 0x28B4, 0x3278, 0x328C, 0x32B4, 0x32DC, 0x3C64, 0x3C78, 0x3C96, 0x3CA0, 0x3CB4, 0x3CC8, 0x3CDC, 0x4650, 0x5046, 0x5050, 0x5064, 0x5078, 0x508C, 0x5096, 0x50A0, 0x50B4, 0x5A78, 0x6428, 0x6464, 0x648C, 0x783C, 0x7864, 0x7878

enemy values:
0x00, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0D, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x1C, 0x1F, 0x21, 0x2F, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, 0x3D

[Naulahauta]

How are you sure those are all?
And most importantly, what do they stand for?

[Commando125]

I checked the header of each level and wrote them all down. Then I sorted the data of object values, enemy values, and 16-bit words, and sorted the lists, then removed duplicate elements.

The first set is for objects such as pillars, escalators, ice, spikes, etc. (Bottom row of 7 unknowns)

The second set I don't know. (Top row of 8 unknowns)

The third set is for enemies such as fish, bosses, salamanders, Umihara, and backpacks  (Labeled in header editor)

For object values, there are 7 spots it can go in. Each spot I think affects tiles $01-$07, $39-3F, $41-47 respectively.

I'm going to work on the enemies first.

object values:
0x00: nothing
0x04: escalator slow right. animates palette 7.
0x05: escalator slow left.
0x06: escalator fast right
0x07: causes death
0x08: rising platform series (bank 0)
0x09: falling platform series (bank 0)
0x0B: pullable/moveable wall
0x0C: door
0x0E: downward pillar
0x0F: upward pillar
0x10: pop-out ceiling tile
0x11: pop-out ground tile
0x18: bridge piece
0x19: bridge piece. collapses when touched.
0x20: single platform (bank 0, white)
0x21: single platform (bank 0, blue)
0x22: waves can push you on platform $41-$47. slippery.
0x26: rising platform series (bank 0, blue, way less frequent)
0x28: boss door barricade
0x29: causes death
0x2A: ice/glass
0x2B:
0x2C: rising platform series (bank 4)
0x2D: falling platform series (bank 4)
0x2E: rising spiked platform series (bank 4)
0x2F: falling spiked platform series (bank 4)
0x30: rising platform series (bank 2)
0x31: falling platform series (bank 2)
0x32: rising spiked platform series (bank 2)
0x33: falling spiked platform series (bank 2)
0x34: single platform (bank 1)
0x35: single platform (bank 5)
0x36: rising platform series (bank 1)
0x37: falling platform series (bank 1)
0x38: single platform (bank 6)
0x39: single spiked platform (bank 6)
0x3A: rising platform series (bank 5)
0x3B: falling platform series (bank 5)

word values:
0x0000: nothing
0x14FF:
0x1E96:
0x1EA0:
0x1EB4:
0x2864:
0x2896:
0x28B4:
0x3278:
0x328C:
0x32B4:
0x32DC:
0x3C64:
0x3C78:
0x3C96:
0x3CA0:
0x3CB4:
0x3CC8:
0x3CDC:
0x4650:
0x5046:
0x5050:
0x5064:
0x5078:
0x508C:
0x5096:
0x50A0:
0x50B4:
0x5A78:
0x6428:
0x6464:
0x648C:
0x783C:
0x7864:
0x7878:

enemy values:
0x00: nothing
0x01: "Poof"-sprite, crashes alone
0x02: "Splash"-sprite, crashes alone
0x03: net spawner (byte before this is what's dropped)
0x04: goldfish
0x05: eel
0x06: flying clam
0x07: brown snail
0x08: bucket, type 1 (byte after this is what's spouted)
0x09: bucket, type 2 (byte after this is what's spouted)
0x0A: flying fish (for bucket)
0x0B: small fish (for bucket)
0x0C: UNUSED. Puffer fish.
0x0D: blue fish
0x0E: "Poof"-sprite, crashes alone.
0x0F: red egg-laying fish
0x10: shark (byte after is what's thrown)
0x11: shark bait, also can be on ceiling
0x12: giant tuna fish
0x13: squid (horizontal) (byte after this is what's spitted)
0x14: squid (vertical) (byte before this is what's spouted)
0x15: squid ink
0x16: tiny fish in water (decorative)
0x17: brown bird
0x18: blue bird
0x19: "Poof"-sprite, crashes alone
0x1A: "Poof"-sprite, crashes alone
0x0B: "Poof"-sprite, crashes alone
0x1C: barnacle (ground) (byte after this is what's spitted)
0x1D: UNUSED. barnacle (vertical) (byte after this is what's spitted)
0x1E: UNUSED. blue barnacle, spits cannonballs
0x1F: barnacle spark
0x20: UNUSED. blue barnacle
0x21: blue snail
0x22: UNUSED. octopus (identical?)
0x23: UNUSED. catfish
0x24: "Poof"-sprite, crashes alone
0x25: "Poof"-sprite, crashes alone
0x26: "Poof"-sprite, crashes alone
0x27: "Poof"-sprite, crashes alone
0x28: "Poof"-sprite, crashes alone
0x29: "Poof"-sprite, crashes alone
0x2A: "Poof"-sprite, crashes alone
0x2B: "Poof"-sprite, crashes alone
0x2C: "Poof"-sprite, crashes alone
0x2D: "Poof"-sprite, crashes alone
0x2E: "Poof"-sprite, crashes alone
0x2F: backpack (1-up)
0x30: giant tadpole (boss) (put 0x31 and 0x36 after this)
0x31: frog (for boss)
0x32: giant seahorse (boss, enemy #3) (put 0x33 and 0x35 before this, 0x34 after this)
0x33: vertical seahorse (for boss)
0x34: small seahorse (for boss)
0x35: giant ink (for boss)
0x36: frog egg? (for boss)
0x37: giant crab (boss) (put 0x38 and 0x3D after this)
0x38: crab's bubble (for boss)
0x39: tiger flying fish (boss) (put 0x3A after this)
0x3A: tiger flying fish second formation? (boss)
0x3B: flounder flying fish (boss) (put 0x3C after this)
0x3C: flounder flying fish second formation? (boss)
0x3D: crab's platform shrapnel (for boss)
0x3E: "Poof"-sprite, crashes alone
0x3F: "Poof"-sprite, crashes alone

[Naulahauta]

The enemy bytes seem to correspond to what I figured out in the Wiki page. Check it out, there's unused enemies too!

[Commando125]

Didn't know about that. Thanks. I updated the enemy byte information. It is going to take a while to figure out the objects because using the wrong bytes when spawning objects crashes the game, so I have to check them one at a time under certain conditions.