Tuesday, December 11, 2007

KDE & DTrace

At the moment I'm looking at adding dtrace probes to Apache/RogueWave's libstdcxx ( the C++ standard library) and I'm coming up against a couple hurdles, not least of which is the requirement for C++ name mangling.

C++ implements templates, namespaces, function overloading, inheritance and a myriad of other things that make plaintext names for functions unfeasible. So, in order to properly probe a function, we can't use the standard provider:module:function:probe naming scheme ( since the function will be something meaningless like
__1cEswap4Ci_6FrTA1_v_ for a function named 'swap'

My current thought is
that, since the function name is meaningless, perhaps we ought to exploit the probe names.

Talking with Damian on irc, we came up with a naming scheme for our probes that looks something like this:
namespace_[class, if any]_function__probename

So a function like will have a probe named 'entry' that looks something like global-swap-TT-TT-entry.

The other option is to pretend the problem doesn't exist, use the fbt provider as normal, and pipe dtrace -l through c++filt but I don't really like that.

Thursday, November 1, 2007

Indiana

The proverbial cat is out of the bag now, and the Indiana project has released something.

I downloaded it last night and installed it in Fusion and Parallels. The installer is still pretty braindead, but I'm sure it'll improve to have options eventually. It is alpha after all.

I ran in to a weird bug where I haven't managed to coerce it in to letting me log in, so aside from that I haven't had a chance to fiddle enough to pass judgement on it.

It unfortunately had to come with what I perceive to be a slap in the face to the OpenSolaris community, when it was pronounced from on high that it shall be called OpenSolaris to the exclusion of all others.

This really gets my goat, so to speak
(I'm rocking the animal metaphors today I guess) I don't particularly agree with the name change to begin with. I think OpenSolaris can exist as a noun in and of itself much as Linux does today, or as Joerg put it, like a screwdriver that you build things with, and not a particular screwdriver.

Ian's ranting about being confused how to download "OpenSolaris" notwithstanding (it being nothing but pure rhetoric to serve a purpose anyways), I think the target audience is intelligent and well-accustomed to the idea of distributions by now that calling OpenSolaris a codebase and nothing more isn't a real issue.

The slap in the face part comes primarily from the way that the name was chosen. Had there been a vote which suggested that Indiana be called OpenSolaris, then fine, fair enough. In this case however, the name came from executive fiat from Murdock quite aside from how the community feels about the issue.

Fortunately the OGB seems to have collectively grown a pair (individual members have been outspoken already. They're good.) and seem to be seriously discussing condemning this action by Murdock and the rest of the marketing crew and imploring Sun to hold a vote on the matter, which is something I support 100%.

So, in conclusion
Congrats to the indiana team for pushing something out the door. The marketing team's actions aren't your fault.

Friday, October 26, 2007

Large cats, ZFS and you

Like many, I got my copy of OSX 10.5 "Leopard" today to install on my MacBook. The upgrade went relatively smoothly and I was presented with the new UI in all it's flashy glory.

After doing the standard explorations: Coverflow in my images/ folder, new Mail.app. OSX supports ACL's now, that makes me pretty happy. Then, I decided to start playing with the OpenSolaris-derived features.

Dtrace. works like a charm. Colloquy spends an inordinate amount of time in syscall::read. A lot of Mac apps don't use syscall:::, that's interesting to note but not terribly surprising considering the way XNU is designed.

ZFS. Now, the version of ZFS that ships with OSX is read-only. My server, an Athlon64, and my workstation, a Blade1000 run SX:CE b74.

So, I pull out my USB stick, put it in the SPARC, and turn it in to a zpool named 'test'. export.
Put it in my MacBook.
$ sudo zfs import -a

The ZFS version is too new. OSX can't import it.

So I decide "what the hell..." and I log in to my ADC account, download the read/write beta of ZFS from Apple. Install it. Reboot. Put the USB stick in.
$ sudo zfs create newpool disk1

'newpool' shows up on my desktop. So far, so good.
$ sudo zfs create newpool/test

still good so far. I copy a small image over to the new pool, it does what you'd expect. So I unmount the volume. Won't unmount because it contains other volumes. that's silly, but okay... it's a beta and OSX doesn't really grok ZFS yet. Fire up Terminal.app again
$ sudo zpool export -f newpool
no dice, dataset busy. unmount it's mountpoint then. Try again. No feedback, it must've worked.
yank the USB key. The Mac gives me the multilingual kernel panic window ( it's not a blue screen, that means it's better than Windows, right? ). Whatever.
I plug the key in to my Blade1k and run a
# zpool import -a
I/O error reading dataset 'test'. Kernel Panic. That's interesting... shouldn't the old dataset been destroyed when the new dataset took the device over?

So, I bring it back over to the Mac and run disk util, maybe I can just destroy the EFI partition table and try again from scratch. Disk util somehow mounts 'test' to my desktop and sits there spinning forever. I can't unmount test at all because it's in use. Force Quit Disk Utility. Nothing. kill -9 it. Nothing.
$ sudo umount -f /Volumes/test
could not unmount. yank the drive. kernel panic.

At about this point I decide that my best course of action with respect to this USB key is to find a machine that doesn't support ZFS and format it there, and give up trying to coax OSX to behave like Solaris.

Casualties: one USB key.
Conclusion: When Apple says "beta", they mean it.

Thursday, October 18, 2007

Aftermath

I appear to have caught some sort of nasty California virus on the airplane back that's left me quite incapacitated.

I've managed to wake up this morning, but that's about the extent of my ability.

Sunday, October 14, 2007

Developer Summit: Day 2

Day 2 went well, with the exception of the naming&branding discussion which ended up a little bit heated. Garrett did a GLDv3 conversion talk & there was a talk on moving core Sun processes such as ARC out in the open, streamlining them, etc.

It felt a bit short today. I'm not sure why, because we didn't get out that much earlier, but it did. Not to say today was entirely unproductive though, because it wasn't.

I think overall, this weekend was a good thing. Whether or not decisions were made that wouldn't have otherwise been is irrelevant, I think it was a fantastic idea to get everyone in the same place at the same time so that the community could all meet each other face to face.

I know I for one learned a lot, and I imagine others found some useful information to take out of the conference. It's also done a lot to dispell some of my fears with respect to Project Indiana.

Saturday, October 13, 2007

Dev Summit. Day 1

Today was the first day of the developer summit & I think it went over fantastic. Jesse, Sara, Glynn and the rest of them have done a fantastic job organizing the thing.
Stephen Hahn's presentation on the packaging system was quite informative and did a lot to calm some of my fears of the chaos of network install. After that I left the main presentation room in favour of seeing Garrett's presentation on fixing a driver to include suspend/resume funcionality, which was great, and then proceeded to Sun Labs' room to plan out the next course of action for the PowerPC port.
I'll organize some notes later & publish them

Friday, October 12, 2007

Early morning Chaos time

I'm sitting on the incredibly overpriced wifi at YXS. I should be on an aircraft headed to YVR, but the airport shuttle heard "7am" instead of "6am".

Sara was an awesome help, so my flight was changed around and instead of YXS->YVR->LAX->SJC I'll be flying straight from Vancouver to SFO and meet someone at the Sun Menlo Park offices.

Dealing with problems at this hour is never fun. Thank you so much guys...

Saturday, October 6, 2007

Closer and closer

nightly(1) now successfully compiles through my code on x86.

It dies with the linker now, which is a vast improvement over the previous situation.

I want this done, debugged & integrated or at least the ARC process started by Christmas, if only to be able to give it as an xmas present to OpenSolaris.

Thursday, October 4, 2007

Status, conferences and the likes

So, In case anyone was curious, nightly manages to build most of emancipation-gate's libc, but I've been monkeying around with it because evidently O/N demands warning free code, and that's causing my code to not build. Lots of it comes from FreeBSD where they use gcc, for one, and are less strict about the build throwing warnings (especially in the regex stuff)

That out of the way, I'm excited to be going to the OSol Dev Summit next week, particularly in light of the recent thread on the mailing list which seems to imply a bunch of us want to break off from the packaging/install/community stuff in favour of more hard technical discussions.

Wednesday, September 26, 2007

It's been a while

I know it's been a while since I've updated (or really done anything for that matter), but the school year started so it's been a bit hectic.

Good news though is that I'm no longer building code in a sandbox with ksh for loops, packing it up as libc_i18n, and putting it in place of the old code; Now I've got my code shoved inside an O/N fork, and ON attempts to build it. Emancipation has a new repo at hg.opensolaris.org/hg/emancipation/emancipation-gate if anyone wants to check it out. One caveat though is that this is a full ON tree, so it'll be quite large.

It's not building at the moment, and I'm not sure why, but since I just coaxed make(1) in to even trying last night at about 2am, I'll figgure it out and fix it. It's probably just something stupid anyways.

Tuesday, September 4, 2007

And, after a week off

I took the last week off because I'm no longer time constrained & it's nice to take a break every once in a while.

last night, I removed a chunk of the GCC-isms that were preventing my code from compiling with studio and removed them ( __inline and __restrict were the main culprits ), so now it ought to work with both compilers, given gcc is passed the correct flags to comply with the standards in question. The code's in the Mercurial repo.

I was going to try to get it to link in this week, but it occured to me that it'd probably be a much better plan to shove it in an ON branch, so I think I'm going to play with makefiles for a while & remove the stupid compile step i've got going on right now.

cheers

Sunday, August 26, 2007

Repo?

I've had a number of people ask me about where exactly to find the code so far.

As of this morning some time, I migrated my code from cr.opensolaris.org over to the Project Emancipation Mercurial repo.

You can create a local clone with Mercurial. If you haven't ever done this before, do this:

$ hg clone ssh://anon@opensolaris.org/hg/emancipation/libc_i18n

and you'll have a local copy of the code

Monday, August 20, 2007

End of GSoC, but not of emancipation

Well, so it's the 20th. Which means, as of noon today, PST none of the work I'm doing counts towards the Summer of Code.

I've got my code building, although it still won't link completely with libc. The linker doesn't see fit to recognize that I do, in fact, have getwc() and friends in an object file.

Regardless, I'm not going to be satisfied until this code integrates with OpenSolaris, so the end of Summer of Code really doesn't mean that much.

Thursday, August 9, 2007

new Toys & Progress

I got this machine on account of I need to break libc in fabulous ways & I need to make sure that it works on both Solaris platforms.

Tons of thanks to everyone that helped me find some SPARC kit to break!

As for progress, I tried to build my code in to O/N last night & it failed on symbol duplication errors, which is about the best error I could've gotten from the build.

I'm going to resolve that & see if I can't get a build of O/N that at least compiles with my code in it by Monday, and move on to testing from there.

Monday, August 6, 2007

Almost...

I spent the last week visiting family in Vancouver, so I didn't do anything last week.

Regardless, with the exception of some private functions that I don't see in use anywhere, I think I'm pretty much done the initial coding.

Now I just need to force it in to libc and put together a workstation to break for testing.

Thursday, July 26, 2007

Mid-program Update

So, it seems my updates are getting less frequent as time goes on, but I'm sure that if you graphed it logarithmically, it'd still look somewhat linear at first glance ;)

Anyhow, The middle of the program has come and gone this week, I've gotten the midterm payment from Google and here's where I stand at the moment:

Most of the functions in my list are complete, with the exception of date/time stuff, which I'll work on next week. I finished off getwc/setwc and friends this week.

The subsequent week ( or next week, we'll see how things go ) I'll be shoving my libc_i18n.a in place of the one shipped with closed_bins and seeing what happens. Hopefully it won't blow up in my face, but that's what testing hardware is for.

Wednesday, July 4, 2007

Halfway there...

It's the 4th of July today, which means happy birthday America to everyone in the country to my immediate south.

It also means that mid-term evaluations for the Summer of Code begin in two days.

Here's my own personal evaluation of myself:

I've got well over 3/4 of the major function groups done ( "function groups" meaning collate, regex, get/set characters/lines, binary is(alpha | number | English | punctuation ), etc ) and compiling cleanly, with some minor testing (albeit not on edge cases) completing successfully. I have a printout of libc_i18n.a's elfdump(1) symbol table output that I've been crossing off with a Sharpie, and it's getting more and more black by the day.

I feel as though I can complete the coding in the next few weeks, spend the remainder of the program testing & doing the minor tweaking required, and then work on integrating the code to the main Solaris tree ( which, as I understand it, is not necessary for the Summer of Code, so I didn't budget time for it )

Friday, June 22, 2007

hmm....

I think I got strcoll & friends working this week...

It just loops through the string comparing each character and returning 1 or -1 if they're different as per the manpage, but I dunno, it seems like it should be more complex than that

Friday, June 15, 2007

weekly report

All the low-lying fruit is done, as it turns out I could steal most of the EUC stuff from FreeBSD too. Still no idea what the "dense" functions are for ( obviously something densely packed ) but I'll keep looking...

What I have left are things like collate, which I plan to work on this weekend and a week or two afterwards, then regexes (how would you properly pluralize that anyways? ) for a little bit, then getwc & putwc and friends, and then I should be more or less done, save for shoehorning it in to the current Solaris framework and making sure it all works properly.

Perhaps a bit optimistic with the "I've only got to do..." type language, but I'm fairly confident that the project should be completed mostly on time.

Saturday, June 2, 2007

Beginning of program update

So, the Summer of Code program has now officially begun ( didn't stop me from writing stuff before then, but there have you ).

I've written a couple more functions & I've been testing stuff as I go along, no real exciting breakthroughs or any such thing. Things seem to be working more or less as expected ( though changing locales just explodes in a fire of glory )

Cheers

Saturday, May 19, 2007

growing...

I've successfully migrated FreeBSD's internationalization code outside of their tree, and I went to do a symbol count today ( from elfdump(1)'s symbol table ).

I've got 269 defined symbols, libc_i18n.a has 287. Given, FreeBSD has a bunch of functions for fiddling with their "rune" data structure, so as a progress bar the number comparison is more or less useless, I just found it mildly interesting

Unfortunately it all depends on gcc at the moment, and I'd like to get it cleaned up to compile with SUNWspro, but a Studio cleanup is at the bottom of the queue for now.

Saturday, May 5, 2007

Weekly Status

So, I figure it's Saturday, and I should probably make an update as to my progress thus far.

I've got all of FreeBSD's libc/locale subsystem compiling outside of the fBSD tree with the exception of localeconv.c, nl_langinfo.c & setlocale.c which I will finish for tonight or tomorrow evening most likely.

That was all mostly just a bunch of boring monkey work, but it'll make work later a lot easier. I've also been pondering about the best way to link together these objects as something I can encapsulate within a larger library that'll export the symbols I actually need (my linker-foo is weak).

I also need to inform myself about a way to convert the locale on-disk format that Solaris uses in to something that these fBSD functions expect. Since I'm almost certainly not allowed to break things, I'm thinking it'll have to be a runtime conversion with some wrapper functions. Given, I haven't looked too far in to things, so I may be completely off-base with what I need to do here.

Thursday, April 26, 2007

first compile

Like I mentioned in an earlier post, I've found FreeBSD's source to contain piles of eminently useful code.

I'm slowly starting to migrate fBSD's locale stuff to a standalone repo, and I've got a couple files to compile without error.

Whether they work or not is a different story, and I would be surprised if they did to be honest, but it is a start.

At this point I'm thinking that the best course of action is getting the proverbial square peg of fBSD's locale subsystem working on it's own, and writing wrappers to fit it in to Solaris' similarly proverbial round hole with proper type mangling first, and when that's working, move on to the rest of the libc_i18n functions that FreeBSD doesn't implement.

Sunday, April 22, 2007

BSD for the win

Dear OpenGrok group:

Thank you.

I set up an OpenGrok instance on my home machine that indexes NetBSD & FreeBSD, and started searching.

As it turns out, there's a lot in there that I can take, at least as a basic framework anyways, and modify to fit Solaris' locale formats

Friday, April 20, 2007

minus one distraction

I had my last final exam today, and studying for exams was mostly what's been slowing me down.

Now that that's over with, I should be able to work far more regularly. I still have work to contend with (since at the end of the summer I'll still need a job, so I can't just quit), but that shouldn't take up the bulk of my time, and I can be pretty good about asking for less hours if it does start to interfere.

I took a look at NetBSD & FreeBSD's sources for implementation details, and they both look fairly promising. I should theoretically be able to pull a bunch of code out of there.

stay tuned.

Monday, April 16, 2007

Progress

I started actual coding work today, I seem to work better when I sequester myself in to little cubicals in the library ( did I just actually say I like cubicals? ).

wcwidth() is now complete (minus testing) and wtype() is nearly finished (whenever I discover /createthe correct enums), which isn't much but it's a start, and wtype() leads to a whole bunch of other functions ( isalpha(), for example )

Sunday, April 15, 2007

Accepted!

So, I've created this blog because I was accepted to participate in Google's Summer of Code program, which is awesome!

The project I'll be working on is to re-implement OpenSolaris' internationalization functions that are in closed_bins at the moment, meaning they're not open-source.

The reason I chose this project is because one, I believe in (Open)Solaris, I think it's an unbeatable platform, and two, because I believe in open-source.
I think OpenSolaris will really begin to take off when it no longer relies on Sun Micro. to build. As great a company as they are, I can see how people can get scared off by the fact that there are still closed binary components and you can't really build an OSol distribution without the bi-weekly closed binaries.

I'm not hip to this whole blogging thing, but I'll try to remember to write an entry fairly regularly through the summer, to sort of document the whole experience.