ダニエルの冒険

Since last night I’ve written two little applications. I should really be focussing on the Software Architectures project but I really wanted to bang these out. I’ll publish them once I have them polished up a bit and package them.

The first allows you use the “Quick Launch” bar (beside the Start menu) remotely. There are often days when I’m away from the home PC (Honeybee) but want to start a torrent on it. I can remotely start torrents through Azureus but I don’t keep it running all the time, so I need to start it remotely. On OS X I could just SSH in and do “open” on the .app, but it’s not so simple in Windows. Remote Desktop is another option but it ends up messing up my icons (because the resolution of remote screens is definitely smaller than what I use on the desktop).

So there are two parts: a server which sits in your system tray and a client which gives you a list of remote machines (or allows you specify a host/port manually).

Here’s the client’s connection dialog:

When you connect to the remote host, you get a list of buttons. The image for each button is retrieved from the Quick Launch menu’s link’s targets, and sent to the client. Tooltips show the title of the Quick Launch item.

Clicking any of these buttons launches the application on the server. I will keep the server running on Honeybee 24/7, and run the client from my OS X laptop/UL Windows+Linux machines. The only thing left to implement is the all-important authentication so randomers can’t launch any app in my Quick Launch

The second app that I threw together this morning is to display subtitles with manual timing. I got Japanese subtitles for Attack of the Clones but the timing is totally messed up (possibly from a different edition of the DVD), so I need to control the subtitle playback as the movie plays. I wrote an app that basically just allows you pick a file, puts a black bar at the bottom of your screen, and shows each line of that file as you hit any button on the keyboard. Left-arrow will step back and escape will quit but any other key moves forward one. Here it is at the start of Attack of the Clones:

The hypothetical file-system I described in a recent post is also becoming a reality. I wish I didn’t have exams and projects to interfere with all this productivity!!

Until today’s return to gloomy weather I really thought I could start listening to bright summer music. I guess Sazan’s Umi no Yeah!! album (one of my favourite summer albums) is going to have to wait a while longer.

Check out 旅立ち (Tabidachi) by Funky Monkey Babys as an example of some cheerful summer music!

Sazan’s 勝手にシンドバッド (Katte ni Sinbad) from Umi no Yeah!!:

This post is really just a reminder for myself to investigate this idea when I have time. I’d really like to have a go at this project if I ever get the chance.

What I’d love to have is a distributed file-system that’s based on some P2P protocol (like BitTorrent) where files are broken up into blocks, and those blocks are sent to different computers (storage nodes). One computer (tracker) will keep an index of where all these are. AFAIK, BitTorrent is not a great protocol to use for read/write filesystems, but my interest is in a write-once FS.

Ideally there would be some FUSE-based front-end so that a user could mount a folder to provide read/write access to the FS. When a computer (a client) wants to put a file into this distributed file-system, they drag-and-drop it into the folder and the application breaks it up into blocks, asks the tracker where to send them, and shoots them across to different storage nodes. To get a file, the user can open it or copy it to the local disk, and the application will read the blocks from whatever storage nodes have them.

The tracker is the first obvious point of failure here. I guess each node could function as a tracker and they could synchronise data among themselves. The tracker(s) would have all the responsibility, as the storage nodes would just receive, store, and send blocks, and the clients would just read blocks. If there is any level of redundancy, the tracker is responsible for instructing nodes to send blocks back and forth to each other to the point that the level of redundancy required is satisfied. I don’t think I could afford to have redundancy in my home network, and if that’s the case, then I’d like to have a “decommission” operation available. This would remove a storage node from the system. The node would transfer all its blocks to other machines and then cleanly remove itself once it is no longer needed.

Based on my googling this afternoon there seem to be quite a few distributed filesystems available, including some older ones like GFS, some inept ones like Microsoft’s DFS, some “community-based” online ones like wua.la, and some awesome-but-not-ready ones like Ceph.

What I really want is one that is:

• Multi-platform: I want to use every machine in the house so Windows XP/Vista/OS X Tiger+Leopard/Ubuntu Linux.
• Local: I don’t need/want to communicate outside my LAN. It’s slow and, to a lesser extent, means I need to think about security. I also lose some level of control.
• Free: I’m really really cheap.
• Lightweight: I don’t want to have to install Active Directory or Linux or anything. Coda is almost entirely in Python and doesn’t need to be installed—nice!
• Compatible with commodity hardware: I just want to use the hardware available. That’s an ethernet network, consumer-level PCs/Macs with consumer-level operating systems, and a couple of hard drives in each.

I’d really love to get hacking on this…

The FYP is finally complete! Weighs in at 28,465 words.. 135 pages printed (106 pages excluding title/TOC/etc and appendices).

Report (PDF, 2.06MB)

Release page here.

Release coming later on today. In the meantime, here’s a walkthrough of the game.

I just went to Tilmitt’s house to check out how well YakumoDemo would work (or if it would work at all) on a non-development machine.

At first, it was just crazy slow. When adding an object to the sphere, it would lock up for about a second or two. The framerate was averaging about 1.3fps. I opened perfmon to see what was going on, and saw that CPU usage was about 24%, but that figure was for all the cores. When I opened taskmgr I saw that the first core was being totally choked. But what was even more interesting was that when it started to get choked, YakumoDemo’s screen went black, then grey, then totally locked up.

It turned out that when taskmgr was active, it would always freeze when adding a new object to the sphere! I tried changing the affinity of the process to use only two of the four cores on his system and it was performing fine, just like at home. It chugged a bit but no real problem. Then I changed it to use only one core, and it was perfectly fast!! Keep in mind that all the threading code was totally unchanged.. the thing is still using about a dozen threads, but by throwing out all the other cores, it was dramatically faster! The only explanation I can imagine is Tilmitt’s: Vista’s threading model ends up moving threads between cores a lot. If there’s a lot else going on, maybe it takes a thread off a core and lets some other process use it, then tries to find a new core for the thread, which takes time? I don’t know…

Anyway, here’s the last-minute hack I put in to fix it:

// Due to some problem with either the Vista threading model,
// Havok's multithreading, or both, everything currently runs
// faster if only one core is used.
{
	// Find the number of cores available on this system.
	SYSTEM_INFO sysinfo;
	GetSystemInfo( &sysinfo );
	int cores = sysinfo.dwNumberOfProcessors;
	// Set this process to use only the last core.
	SetThreadAffinityMask( GetCurrentThread(), cores-1 );
}

Yesterday was Demo Day and apart from a lot of setup problems it went really well. At first, the monitor kept displaying everything in yellow, and I couldn’t get the iPod on the same subnet as the PC until I got my own wireless router set up, but after that it went pretty smoothly. I’m still recovering though: my back is sore from dragging around the PC/laptop/monitor/wireless router/bag of cables and I still haven’t recovered from this week’s sit-a-thon. Also, last night when I was trying to go to sleep I kept getting images of that big green ball rolling around with various objects attached to it though, and this morning when I knocked over some bottles of shampoo in the shower, for an instant I thought that the movement was slightly unrealistic. I made a mental note to try changing the shape radius of the bottles… and then the instant was over and I reminded myself that I don’t need to evaluate how real every physical interaction I see is anymore.

The most enjoyable thing yesterday was seeing people get really involved with the game. Because the player’s objective is to grow the size of the ball, they seemed to get an emotional involvement in that ball. I always felt the same with Katamari Damacy and it was really rewarding to see other people reacting the same way. Two people played it through to the end, and many others went as far as they could. Unfortunately some of the early players couldn’t actually finish the game because I hadn’t fully finished tweaking it, but as players went as far as they could, I would make a note of what was stopping them and make the change for the next player. for example, for an object to attach to the sphere, it must be 0.006 times the sphere’s volume. I changed this to 0.0064 and then people were able to attach to the milk/juice cartons and clear up the whole room. I also saw cases where a player had picked up everything they could, but the volume ratio was about 0.0061 or even less, so I added a couple more gameboys to the room.

One thing that might invalidate some of the playtesting is that I was playing a pretty big part in the gameplay. It was more like Dungeons and Dragons than Katamari Damacy, in that sense. If I felt that a player was getting discouraged, I helped them by directing them to the next goal, or reminding them how close they are to being able to stick to all those bananas (“just get those two matchboxes and you should be able to roll up the bananas and apples and then you’ll be well on your way to finishing it!”).

My second reader is just as much a fan of Lafcadio Hearn (Koizumi Yakumo) as I am, so he was pretty excited even just about the name “Yakumo”. Lots of other people seemed interested too, and even though I was in the very corner of the room I seemed to have a bit of a queue sometimes. One or two of my friends had to wait about 10 or 20 minutes in line to get to talk to me! The game itself seemed to take about 10~15 minutes to complete, and there were some interesting questions.

Actually, about questions: I didn’t feel like I had to do much presenting or that people had many questions. As I was hoping the game seems to stand pretty well by itself. I just explained the controls (move the iPod to control movement, move your finger on the screen to tweak the camera) and people were happy to play the game as it was. There weren’t many suggestions for what could be done to make it better (which is possibly a good thing!) and there weren’t many questions about how things worked. For example, I only got to explain my camera workings to one person, and it’s the same story with how objects stick to the ball.

When asked how much/what sort of work was involved it was difficult to decide where to start. The subsystems which tie together the various libraries? The YkRemote library (library for iPod interaction)? How the gameplay is defined in code? How I developed the thing (Subversion, blog, etc)? The development system (Xcode w/ gdb over the USB cable for the iPod, VC++ for the PC, Havok VisualDebugger for the physics.. all over two or three computers)? The content pipeline (creating assets in Paint.NET/3ds max and how they end up as interactive objects in the game)? There’s just so much to talk about. Not a bad thing considering I’ve so much report-writing ahead of me though!

So much report-writing…

Looks like my ridiculously poor hkpListShape structure has come back to haunt me. I forgot that the max number of shape keys you can have is 8, so after reconstructing the shape 8 times, everything fell apart as soon as I tried a shape cast.

Now instead of just replacing the old list shape with a new one containing it plus a hkpTransformShape, I’m properly rebuilding the whole thing and it seems to be working!

Very late to be worrying about this sort of stuff! I wonder how many bugs I’ll find tomorrow?

Turns out it takes ~85ms to convert the PixelFormat for a PixelBuffer. Using a RenderTarget’s suggested PixelFormat (RenderTarget::suggestPixelFormat) means you don’t have to do this. Doubled the speed of framebuffer->PNG code