JPype : Chronicles of an Open Source project

JPype is alive! kinda ....

2007-09-23T10:43:00.000-04:00

Despite a few false starts, I am still working on JPype. I haver recently undertaken to rewrite it again.

Last year was a major false start. The architectural goal of writing less C++ landed writing just as much (in glue code), but also write a lot more of glue java and python code. Best forgotten.

This time around, I decided to tackle the main issue first. So after doing minimal work on general class support, I tackled what I set out to do all those years ago : subclassing.

As of this writing, it works! With a few restrictions, it is possible to extend a java class in python, pass an instance of it to java code and have the java code call the python methods! Hurray!!!

With this release, I have also decided to get serious about multiple host language support. To this end, the main package has been renamed to JEmbed. It has a completely new API geared towards the new functionality.

JPype will continue to exist. My goal is to re-implement the JPype API on top of JEmbed such that existing programs will continue to work as-is.

I am also going to try an open approach to the progress. All the main tasks are listed on t he sourceforge project (JEmbed and JPype ). As things progress, I will add tasks there, and track their completion.

If anywone sees anything missing from there, or have requests to make, let me know! please open a feature request on the site ( http://sourceforge.net/tracker/?group_id=109824&atid=655015 )

Thats it for now.

Striking a balance

2006-10-30T22:24:00.000-05:00

Sometime while I was writing version 0.5 of JPype, I realized I was writing a lot of C++ code, so I could enoy writing in Python ... not a very intelligent thing to do I think.

So in version 0.6 I decided to move as much "logic" from the C++ layer, up to the python layer or down into the Java layer, depending on the actual functionality.

0.6 is well under way, and so far thigns seem to be holding up. The upshot od that both Pythona nd Java are a LOT easier to work with than C++, so development velocity is greatly enhanced.

There are 2 caveats though. Hence the "striking a balance" title of this post.

The first is that with logic moved outside of C++, the interface between Python/C++ and Java/C++ becomes that much more complex. Handling all that by hand would mean a lot boilerplate code, and a lot of potential mistakes. So to ease that burden, I've written two tools : one called JNILayer is for the Java/C++ interface, the other for the Python/C++ layer is called PNIH (Python Native Interface Header). I'll probabl talk about them in a later post.

The other caveat, also related to the more complex interfaces, is performance. JNI is known as a performance bottleneck, and I can't imagine that the Reference wrapping I've had to do helps anything. Hopefully, performance will not be abysmal, and with a few tweaks will be good enough to be often by the other benefits.

I guess only time will tell :)

Shelving PNI for now

2006-10-17T09:43:00.000-04:00

I realized that PNI, while a good idea, will require a lot of time and effort to realize. So for now I have decided to concentrate on JPype. The work I did on PNI has already impacted the release of the next version of JPype, and I don't want to impact it more.

I am still developing JPype with PNI in mind. The only non-pni-compliant code in JPype is auto-generated. So if PNI ever comes to be, I can simply change the code generator (and of course adjust the rest of the code to whatever final form PNI has).

If PNI never sees the light of day ... Well then it'll simply be a abstraction layer that JPype uses :).

Is the python community unable to see it's own weaknesses?

2006-10-10T16:21:00.000-04:00

Over a week and a half ago, I posted a proposal on the Python mailing list/newsgroup. If you've seen my previous blog entry you already know what it was about. For the others, it is a proposal to remedy one of python's weaknesses : native extensions must be compiled for a specific Python version using the same compiler used to compile that python version. I've seen over time many people call this a nightmare, especially on windows.

Fast-forward one week, and I haven't had a single response. Not one! People on the mailing list wax poetic on trivial issues such as being more polite toward Perl.

Is this a case of blindness? Are extension writers so comfortable with this situation that they don't care? I mean I don't expect my proposal to be accepted as is, but I was hoping it would at least spark a discussion.

I will assume the answer is different, and that it is a simple case of bad timing. I will prepare a shorter, easier to digest proposal and post it again. Then we'll see.

In the meantime .. Feel free to tell me how receptive the python community at large has been to your ideas so far.

PEP proposal : Get rid of the version/compiler dependency

2006-09-29T15:26:00.000-04:00

I am creting this post so people can leave comments on the proposal I posted. The proposal can be seen at JPype's home page (http://jpype.sourceforge.net/pni.html).

JPype success and horror stories

2006-09-27T23:37:00.000-04:00

I was browsing the net recently, and thought it might be funny to google for "jpype". Boy was I surprised! Google reports nearly around 18400 hits!!!

Granted, most of those hits are archived repeats of usenet posts made over a year ago .. But still, I can see the interest.

What would like to know is who uses JPype. What projects. Why tried and failed (and why!). I'd like to make a page for JPype website with links to those projects.

It's always nice to see that my had works is actually benefiting someone else.

A bit of History

2006-09-20T21:07:00.000-04:00

For those wondering, here's the history of JPype :

I always have a lot of projects going on. And in many cases, while I would prefer to use Python to implement them, requirements and/or convenience often steers me toward Java. Let's face it, when it comes to community mindshare, Python is no slouch, but Java definitely is the 500 lbs. gorilla.

But I really wanted to use Python, so I looked around to see how easy it was to mix the two. Jython (JPython at the time) was not an option because of general slowness and lack of feature support. I failed to successfully build the only python/java integration library I found. So I decided to build mine. That was back in may of 2004.

The initial versions (0.1 to 04) were more or less of prototype quality. The C++ code was extensive, with lots of Python extension type and lots of problems making Java classes behave like python classes. Java-specific code and Python-specific code were hopelessly locked together.

0.5 was a complete rewrite, with an eye towards separating the bridging code. Although the amount of C++ code didn't shrink, this saw the introduction of real, dynamically create, Python classes. No more trying to make extension types behave like regular python classes. This was almost perfect.

Major limitations include the inability to raise/except with straight java exception classes (needs to use the member PYEXC instead), and the inability to cleanly shutdown/restart a JVM.

JPype got it's first real test when Chas Emerick of Snowtide Informatics (www.snowtide.com) contacted me about polishing JPype for use in one of their product. I can honestly say the partnership has greatly benefited JPype, with all the improvements made then folded back into the code.

The release of 0.5 has been followed by a lengthy pause in development. Lack of time and interest in other issues being the major reasons. Now time has come to come to resume work towards that almost mythical 1.0 release. 0.6 will be out sometime in the coming months. The details of this, however, will have to be the subject of another post ...

Read back for more info later on.

A new start

2006-09-18T00:08:00.000-04:00

Well, it had to happen someday. Welcome to my first ever blog posting.

I can't promise this is going to be the most regular of blog, but I will do my best.

I am the principle author of an obscure Open source project called JPype.

I thought it would be interesting to share thoughts and ideas that come to me as I develop it. So this blog is going to be very programming-oriented.

Well, that's it for now. More on this later!