Paul D. Eden

How to fix the permGen memory error in Intellij 10.5

2011-06-17T14:53:00.000-07:00

If you were like me, you were working on a large web-based java project and had intellij configured to start jetty for you.

All was well until you tried to start jetty using the Intellij Run or Debug functionality and it errors out every time due to a permgen memory error.

Google showed quickly that the problem was a memory configuration of the JVM, but how to change that configuration was not quickly apparent to me.

So this is how you do it.

Find the JVM memory values you want to change from the stackoverflow.com link above. I used -XX:MaxPermSize=256M which worked in my case, YMMV.
Enter that configuration into the Run/Debug Configurations page in the VM parameters field.

Voilà!

Merging a branch with git-svn and still retaining the association with the previous svn path

2009-09-16T10:47:00.000-07:00

git-svn uses commit messages to tie the commits in a particular local branch to its corresponding SVN path, which works great most of the time.

It causes a problem though when you want to merge branches in git and have the merged results be put back into svn. I wanted to do this because the git merge functionality is much easier to use, IMHO, than subversion's merge command.

The problem is that when you do a git merge of two remote svn folders in a local git branch the commit message path changes from the previous path to the path of the source of the merge.

So, for instance, if you want your trunk in svn to match a branch (let's say branch 1.1.1), you could do the following in a git-svn checked out repo (Assuming you have already done git svn clone -s...)

git checkout trunk
git checkout -b localtrunk

You are now in the local git branch 'localtrunk' which is a copy of the trunk path of your svn repository.
Now, let's say you want to merge changes from branch 1.1.1 into trunk, you could do this...

git merge 1.1.1

If the merge is successful there is still a problem. The problem is that now if you do a git svn dcommit to push the merged changes into trunk they will not go into trunk. They will go into the 1.1.1 branch! Bummer. If you read the man page for git-merge you might say, "Hey, there is a --no-commit option to git-merge! That could fix the problem." In theory you would be right and I thought the same thing, but after many attempts (on Ubuntu 9.04) it would always commit after merging. Bummer! --no-log didn't work either. So, after a lot of searching, I found a way to do this, but it only works if you don't have a lot of commits to replicate.
Instead of running git merge as above do this. For each commit do...

git cherry-pick -e (SHA1 hash of the commit to merge from branch 1.1.1)

You can get those hashes from git log, after doing git checkout 1.1.1 in another terminal.

Each time you do it delete the line in the commit message that starts with git-svn-id: (and the line after it, but I don't think deleting that 2nd line is technically necessary)

Now do...

git svn dcommit (to send the merged changes to svn while keeping your local git branch pointing to trunk!)
git svn fetch
git svn rebase

Check that your localbranch branch and your 1.1.1 branch are the same with...

git diff 1.1.1

If there is no output they are identical.

Check that your localbranch branch is still pointing to trunk by doing...

git log

and verifying the git-svn-id: lines for the last commits (the ones at the top) still point to trunk@(svn commit number)

All done!

How to easily fork python code into separate processes via a decorator

2009-06-30T10:42:00.000-07:00

Do you have a bit of python code that is taking a long time, and you don't want to wait for it to finish?

The blogger here made an excellent way of forking pieces of your code into separate processes using a decorator.

http://buffis.com/2008/01/09/easy-forking-of-processes-in-python-using-generators/

It doesn't get easier than this.

How to get the firefox java plugin working in Ubuntu 8.10 64bit

2009-06-19T03:17:00.000-07:00

The default java plugin for Firefox on Ubuntu 8.10 is not fully functional on 64 bit machines. It works for some applets, but not all.

This blogger took the time to figure out how to replace it with Sun's plugin. Thank you for sharing!

http://tp0x45.blogspot.com/2009/03/openjdk-issue-in-ubuntu-810-64-bit.html

Use Python for Microsoft Windows Login and Group Policy Scripts

2009-04-17T11:14:00.000-07:00

Background

Microsoft's Active Directory provides a way to distribute msi files and execute scripts on login, logout, shutdown etc. This makes it very easy to distribute a script to be run on login for a group of machines registered to Active Directory.

The type of scripts that can be run are .bat files, JScript or VBScript files via Windows Script Host (WSH. It's executables are wscript.exe (windows version) and cscript.exe (command-line version)), or windows executables (exe files). While those options are not too bad, I much prefer python for programming so here we are going to outline how to use python for these scripts.

Tim Golden has written some excellent wrapper code for python to make programming for Active Directory, WMI, shell work (like creating shortcuts) amoung other windows-specific things a pleasure.

Instructions

Get the Python MSI from Active State. We could have used the msi from python.org but we would have had installed the windows specific stuff that Mark Hammond wrote (pywin32 Windows Python Extensions) separately. And since I want to distribute this with Actirve Directory I would have had to convert the pywin32 package to an MSI as it is distributed as an exe. I tried compiling the source distribution to an msi with distutils (i.e.,
python setup.py bdist_msi
) but the build failed because I did not have Visual Studio 2003 installed (I was building python 2.5). I do not want to buy or install Visual Studio (especially the old 2003 version) so I was happy that Active State packaged it all up in an msi already and distributed it for free! Thank you Active State!
If you would like to install Tim Golden's wrapper code via Active Directory as well (highly recommended) it is easy since he provided the setup.py distutils scripts for each package. Just download the zip files you want from his site and then for each one run
python setup.py bdist_msi
and an msi will be created for you. Thank you Tim!
You can distribute any python code that you want via Active Directory this way. Just make an msi of your code by writing a setup.py script and then running
python setup.py bdist_msi
. Distutils makes it easy!
Then, distribute those msi files with Active Directory Group Policy. I'll leave that to an exercise for the reader, but here is a good starting point. For some reason, I had to reboot a couple of times on some workstations for them to get the msi files installed.
Now your setup work is done and your workstations (or servers) have python installed along with pywin32 and optionally some nice wrapper code. From here you can write python scripts for those hosts that will be able to interact with active directory, WMI, COM, etc.
Now write your python login scripts.
To setup your login scripts to run at login you have a choice. First, you could compile your python code into exe files. You can do that without a lot of effort using py2exe (a distutils extension). This route adds a separate step (conversion to exe file) and every time you make a new exe it copies the whole python interpreter and other code with it so the exe files are large. This is a good option if you don't want to install python on the hosts though.
Windows Script Host (WSH) integration is another good option (I won't go into calling the python intepreter on your scripts via a bat file, but that is also an option). This way cscript.exe or wscript.exe runs the python intepreter which runs your script. Here's what you need to do to set this up.
- First, run C:\Python25\Lib\site-packages\win32comext\axscript\client\pyscript.py (adjust for your installed version of python) on each host. This registers python as a language that Windows Script Host can run (called pyscript). Once this is done your python files (saved with the .pys extension can be used as Windows login scripts! You could put this command in a MSI file and distribute it with python so that when your login script runs it will already have been done.
- There is one registry entry that needs to be added I have found in experimenting and below is some jscript code that does it (put it in a file in with a .js extension that you can publish it via a Group Policy in the Computer section of the policy to run on startup of the machine (best not done on logon to make sure it get's done before your python scripts run))
  
  var objShell = WScript.CreateObject("WScript.Shell"); objShell.RegWrite("HKEY_CLASSES_ROOT\\pysFile\\shell\\open\\command\\", "\"C:\\WINDOWS\\system32\\wscript.exe\" \"%1\"", "REG_SZ");
- Lastly setup your new shiny .pys script to run at login user the User side of the Active Directory Group Policy Object. Here is a site that shows you how.
- This site has a lot of good information as well, but I have found that you don't need to do all the steps it outlines because the msi files above do most of it. The steps above worked for me.

edendb - A thin, flexible and fast python DBAPI wrapper for those who like SQL but don't like to type

2009-01-30T13:30:00.000-08:00

If you are like me, you have looked for ways to make connecting to database from your python code easier and less painful.

For a long time I looked into the myriad ORMs available for python and they all seemed to be overkill for me. I don't mind using SQL. In fact I like it, but when putting it into python code and using the DBAPI I found my productivity was declining. I needed something to make executing sql commands faster and easier from python code. I needed something abstracted away from the DBAPI, but that also allowed me to execute straight sql if needed and easily handle all the possible values that could be returned.

So, I wrote a python library to do it.

Meet edendb!

Check it out and let me know what you think.

Google Talk on Ubuntu Linux

2009-01-29T15:23:00.000-08:00

Chat, Voip calls, system tray, Excellent!

http://jamesselvakumar.wordpress.com/2008/11/17/good-news-for-google-talk-users-on-linux/

How to configure mutt to use Gmail as smtp and imap server

2009-01-22T15:13:00.000-08:00

The following helped me the most when configuring this.

http://shreevatsa.wordpress.com/2007/07/31/using-gmail-with-mutt-the-minimal-way/

The only thing I changed was

set postponed =~/Mail/Drafts

so that I could write messages offline and then send them when I get back to an internet connection.

I can't wait to try this out on my Asus Eeepc 2G Surf

Creating an application to insert events into Google Calendar

2009-01-20T14:22:00.000-08:00

Here is how you can create a quick command-line application in python that will post events into Google Calendar.

Get a Google Calendar account (go here to get one).
Install the python gdata client which you can get from here.
Find out the url to use to post to your calendars. To do that just go to this url and then view the source of the xml document that it returns to you. Each of your google calendars will be listed with the url that you need to use for each one. Remove the 'http://www.google.com' part of the url when you use it in your code.
Write up some code to post with. Here is an example.

import gdata.calendar.service
import gdata.calendar
import atom

def add_to_gcal(item):
 cal_client = gdata.calendar.service.CalendarService()
 cal_client.email = 'youremail@gmail.com'
 cal_client.password = 'yourpassword'
 cal_client.ProgrammaticLogin()

 event = gdata.calendar.CalendarEventEntry()
 event.content = atom.Content(text=item)
 event.quick_add = gdata.calendar.QuickAdd(value='true');

 new_event = cal_client.InsertEvent(event, 'url from step 3 above')

add_to_gcal('Dentist Appt today at 4pm')
print "Your event was created"

There you have it.

Pretty Printing XML in Ubuntu on the command line

2009-01-20T14:19:00.000-08:00

Ever wanted a quick and easy way to reformat xml so that it is more readable?

In ubuntu just do this.

sudo aptitude install xml-twig-tools

Then you have a nice little program called xml_pp with which you can easily format xml documents.

Use it like this.

xml_pp document.xml | less

Gears 64 Bit in Unbuntu Intrepid (8.10)

2009-01-17T10:23:00.000-08:00

Gears (formally called Google Gears) does not (yet!) officially support 64 bit builds for linux/Ubuntu.

But, I have found that there has been significant work in the open source community to overcome this shortcoming with unofficial builds/patches.

Here are some links to background.
- http://groups.google.com/group/gears-users/browse_thread/thread/4584842dd14a4882?pli=1
- http://code.google.com/p/gears/issues/detail?id=335

Here is the link to the one that work for me.
- http://dev.laptop.org/~joel/gears/
- I used the latest version as of this writing which is gears-linux-opt-0.5.11+r3162_FIREFOX3_ONLY.xpi
- Works in Kubuntu 8.10 Intrepid with Firefox 3
- When you go to a page that has Gears capability, just try to enable and click the install button (even though it is already installed) and it will skip the installation and enable gears for that page.

Ubuntu 64bit

2009-01-17T10:18:00.000-08:00

My Employer just gave me a new laptop to use with a shiny dual core, 64bit processor!

I installed Ubuntu 8.10 in both 32 bit and 64 bit versions and played with each just to get a feel of which is faster and to see if 64bit would be more trouble than it was worth.

I have to say that I DO notice a difference in speed in the 64 bit version.
- This may be only the placebo affect, but I don't care, it feels faster!

The only downside I have seen to it is there are a few applications that are just not yet available in 64 bit versions.
- The most important of which for me is Gears (formally called Google Gears) as I like to use a few key offline applications (read Remember The Milk).

Next post I'll show you how I got around the gears problem.

T2/T3 web2py cheat sheet

2008-12-31T11:51:00.000-08:00

I wanted a quick reference to the new, high-level web2py functionality that was added in the T2 plugin and the T3 super-powered wiki application.

Here is what I came up with.

http://dl.getdropbox.com/u/169957/t2_t3_cheatsheet_0_9_0.pdf

How to setup pyodbc to connect to MSSQL from ubuntu linux

2008-12-29T20:29:00.000-08:00

Pyodbc is a great python sql db 2.0 interface to odbc. It is a mature and well written library.

It's also a little hard to setup on linux due to all the pieces involved, but never fear, here are the steps on Ubuntu 8.04. The following will show you how to setup the dsn-less setup. This way you don't have to setup DSNs for each connection. Thanks Guillermo for the tip.

sudo aptitude install unixodbc unixodbc-dev freetds-dev tdsodbc python-dev

Change /etc/odbcinst.ini to

[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/odbc/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsS.so
CPTimeout =
CPReuse =

Now unixodbc and freetds are setup.

On to pyodbc itself.

Download the current version of pyodbc

Unzip the file.

Do a

sudo python setup.py install

All done.

Test it with

python
>>> import pyodbc
>>> conn = pyodbc.connect("DRIVER={FreeTDS};SERVER=dns_or_ip_of_server;UID=username;PWD=password;DATABASE=database_name")

No errors and it is installed and working!

More details here and here.

SQLite GUI Linux

2008-12-29T07:25:00.000-08:00

Looking for a GUI for sqlite?

Want it to run on Linux? Ubuntu?

Try sqlitebrowser.

It's in the latest ubuntu repos so just run.

sudo apt-get install sqlitebrowser

How to make a copy of a Table via a TSQL command in Sql Server

2008-12-24T14:24:00.001-08:00

SELECT * INTO <newtablename> FROM <oldtablename>;

How to create a DVD/VCD/SVCD/XVCD slideshow on linux

2008-12-24T06:28:00.000-08:00

I did this at work for some co-workers with lots of NASA pictures.

Make mpeg files that are appropriate for the standard you are using (vcd, svcd, xvcd (not recommended for compatibility with dvd players, but good quality), dvd, etc).
- Use digikam to do it.
- Tools -> Create Mpeg Slideshow
- use NTSC video type for compatibility with US DVD players
- This takes a LONG time

Burn the mpeg files with k3b using the video vcd (works for xvcd and svcd as well) or dvd project setup.

Stackoverflow.com update

2008-12-11T06:49:00.001-08:00

After having used Stackoverflow.com for awhile I must say that I am impressed.

I always thought that having a forum specifically for computer programmers would be a good idea.
But, I am VERY impressed with the implementation.
- The site has an incredible following
- The answers come very quickly and are almost always relevant and helpful
- It's almost faster to ask a question there and get responses than to search google and read through the results even when google gives you the correct answer.

Highly recommended for software developers.

Sarah's New Book - My Favorite

2008-10-09T16:21:00.000-07:00

My beautiful and authorily-blessed wife Sarah has just published her first book on Amazon.com.

It isn't the first book she has published (she published quite a few on lulu.com previously), but we are both excited about this one being published on Amazon because of the huge user base that Amazon has.

This last book; called, "The Price Paid" is a Ghost Story/Romance set in Whales during the Recency Era.

It is my favorite of her books so far! She has written some excellent books before, but I like this one the best because it is so hard to figure out the ending. Anyone who has read a couple of romance books (I don't read a lot of them mind you, actually just Sarah's, but she does read a lot of the clean ones and she has given me the lowdown) knows that the plot is basically the same in all of them. Two people meet, their is something that comes between them, that something is resolved and then they are united and there is a happy ending. I love the plot line. Happy endings are my thing. But this one has such a perplexing twist that you have a really hard time figuring out the end.

Highly Recommended.

Check it out here. The Price Paid by Sarah M. Eden

Stackoverflow.com

2008-10-09T08:21:00.001-07:00

I have been following the development of http://stackoverflow.com for a while now.
It is a joint effort between Joel (of Joel on Software fame) and Jeff Atwood (author of the popular Coding Horror blog).

I had not posted a question until today and I must say that I am pleasantly surprised how helpful it is.

I posted this question and then within minutes I received two excellent and different responses.

I will definitely be using this website more often.

My Book Wishlist

2008-10-09T07:58:00.001-07:00

For those of you who are curious, or would like to give me a present, here for your perusal is the list of books I would love to read.

s3 Pricing drops

2008-10-09T07:30:00.000-07:00

Just when I thought I had the cheapest and best online storage available, Amazon dropped their pricing on S3. Granted, I won't see much of a price change because I don't store TB of information, but man, I like how this is going.

Python Web Frameworks and When I Use Them

2008-10-06T09:42:00.000-07:00

Here is my take on all the myriad python web development frameworks available.

Online Storage AND SYNCING

2008-10-06T09:27:00.000-07:00

After creating my own system for syncing my documents to Amazon s3 (using the great s3cmd (python version) and optionally git) I happened upon a nice alternative.
- For me, I don't use it because what I have works great and I get some sick satisfaction from paying people for the services they give me.
- But I get even more satisfaction from not paying a lot.
- My last month bill for Amazon S3 was 20 cents. :-)

Anyway, the nice alternative is dropbox.
- http://www.getdropbox.com
- First 2 GB free
- Automatically!, and quickly, syncs anything you put in the dropbox folder on your computer to all the computers on which you have dropbox installed!
- Made with Python (I snooped under the hood a little to find this out) which is a plus in my book, not because I can change the code (they didn't include the source, and yes I know you can reverse engineer python optcodes, but I don't want to and it is probably against their terms of service), but because this frees them up to develop new features and fix bugs very quickly.
- Highly recommended!!!

Online Storage

2008-07-17T07:40:00.001-07:00

I wanted to have some files available to me wherever I go. A simple request, but to get it the right way, for me, was not so simple.

I tried a usb drive and that works great for many things, especially for times where I don't have internet access, like at church. (I'm in the bishopric and bring the printed program on my usb drive so it can be printed and copied at church for Sacrament Meeting).

But, the usb drive takes some time to insert open, nativate to, etc. It would seem like a trivial amount of time and sometimes it is, but other times it is just too much time, especially when I am online and working feverishly. I'm a software developer and am online a lot.

So, I looked around for free online file storage services, and while there are a plethora out there, and even for free, none had the feature set I wanted.

A programmatic API
access from Linux, specifically Ubuntu (my home and work desktop OS)
free or very cheap

Here is what I have found and am very happy with so far.

Amazon S3 service for the file storage (which has only a programmatic web services API)
www.s3browse.com for my one-off interface needs. It is a completely free web interface to your S3 account. Very nice! I wonder why, or rather how, it is free though.

Now, I'm free to automate any file storage and retrieval I want when the needs arise and I'll be paying less than $5 a year (estimated).

Mindmapping and access

2008-07-17T07:14:00.001-07:00

Yesterday I tried mindmapping software for the first time.

Wow! Where before I had many huge projects floating around in my mind, now (at least for the projects that I mapped out) I have a calm and in-control feeling. That makes mindmapping something worth doing to me.

I tried first www.mindmeister.com which I liked a lot.
Pluses

online
AWESOME openid login (I highly recommend everyone use openid. It saves so much time and energy with managing logins)
easy to use
Google Gears offline support
looks nice
responsive interface
imports from freemind and others
exports to rtf (even with the free account!!!)

Minuses

You have to use it if you want to save more than 6 mind maps (after the first 30 days)

Being the cheapskate (I like to say, "frugal") that I am I decided to look into other options.
I noticed freemind, which a coworker had been using and recommended. Freemind was great too and seemed to have everything I needed except for the online part.

So, here is what I worked out.

use mindmeister when I need a mind map that is instantly available (I'm tempted to put my GTD lists into a mind map and online would be really nice)
use freemind for project mindmaps that don't need to be accessible from multiple places
put my freemind mindmaps into my amazon s3 storage account using www.s3browse.com (excellent free online interface to the awesome and cheap amazon s3 online storage service)
whenever I need to export a mindmap from mindmeister, I'll export it to rtf, open it in openoffice, and then save it as a text file. Then, I'll use vim or python to change all initial spaces to tabs and then open the file back up in openoffice. Now, a quick copy and paste from openoffice to freemind recreates my mindmap no matter how intricate it is!

I'm happy.