Importing CSV file data into sqlite3

by Mandar Vaze on July 13, 2009
in Code, Hack, Linux, Open Source, tips

The :en:SQLite logo as of 2007-12-15
Image via Wikipedia

I was trying to import the data from CSV file into sqlite3 database. Ideally this should be very simple task, with following the steps given in the sqlite tutorial. It is a matter of calling the sqlite command with separator argument, followed by an import operation, as listed below.

sqlite3 test.db  "create table t1 (t1key INTEGER PRIMARY KEY,data TEXT);"
sqlite3 -separator , test.db ".import some.csv t1"

Except that main attribute of my CSV file was that it could contain single records with embedded comma. I was hoping that sqlite3 would be smart enough to detect that the fields were enclosed within double quotes and then separate by comma. But I soon realized that only a code specifically dealing with CSV would know about this.  As we can see in the example above, the import is a generic code and as a user I listed comma as a separator.

My Data looked something like this :

"1","data1"
"2","data2,data3"

So like any *nix geek would do, I tried providing double quote and comma as a separator. To my surprise it worked very well. I though separator would take only single character, and I had provided two (three?). Anyway, important thing to remember is to escape the single quote with a backslash (I didn’t try it without the backslash, may be that would work too)

So here is the syntax that worked :

sqlite3 -separator \", test.db ".import mydata.csv mytbl"

Update : Turns out SQLite Manager is much better solution after all.  It is an Extension for Firefox and other apps to manage any sqlite database. Not only it took care of above situation, it also handled empty cells as well where the command line failed with following error message :

line 4: expected 3 columns of data but found 2

Data with missing cells : Notice two successive commas :

"1","data1","data2"
"2","data3,data4","data5"
"3",,"data6"
Reblog this post [with Zemanta]

How to Fix the look ‘n feel of GTK Apps in KDE

As all the Ubuntu users know, the default Ubuntu comes with Gnome. Kubuntu is a separate download, focused around K-apps. But I wanted best of both the worlds. It was easy to have both. I just followed the instructions from this page.

Personally, I think KDE 4.x is pretty nice. It comes with host of applications. I think KDE is cool (.. and acknowledge that cool is subjective – personal.)

Since I moved to KDE from Gnome,  I have become dependent on several GTK apps. (GTK is the cross platform widget library used to write the Gnome applications) But by default they don’t look good in KDE.

  1. Firefox tabs would merge into each other, and visually I could not differentiate the boundaries.
  2. Firefox scrollbars didn’t work correctly either
  3. Gnome DO had weird problem, as is the drop-down list which allows me to select the theme went completely bonkers.
  4. The Edit box in Autokey is completely Hidden (See the image Above)

Something had to be done. See the images below. Click on the images to see the details.

Autokey in KDE - Default

The Edit Box is missing

Edit box visible after the dialog is resized

Edit box visible after the dialog is resized

moz_in_kde1

The tab boundaries merged

Follow these simple instructions to ensure that GTK apps “play well” with KDE

  1. Go to System Settings ->Appearance
  2. Click on “GTK Styles and fonts” from the left hand side panel
  3. The very first (at the top) option is GTK Styles. The Default is use KDE Style. Select USe Another Style.
  4. Select Clearlooks (For blue) or Human Clearlooks from the drop down.
  5. You need to restart the GTK Apps.

Now all your KDE/QT apps as well as GTK apps would work well (in terms of display)

See the screen shots after the above fix is applied. Click on the images to see the details.

The Edit box visible clearlt - As designed

The Edit box visible clearly - As designed

Tab Boundaries Visible Clearly

Tab Boundaries Visible Clearly

Reblog this post [with Zemanta]

Integrate Rainlendar Calendar with Thunderbird

by Mandar Vaze on February 21, 2009
in Linux, Productivity, tips, windows

calendar tab

Image by David Ascher via Flickr

So after I started using Rainlendar as my Desktop calendar, now I had two calendars to maintain. One was Rainlendar Calendar, and other was Thunderbird calendar via lightning plugin. It would asoon get cumbersome to update both the calendars. Wouldn’t it be nice if they shared the same calendar ? If they did, irrespective of which calendar I use to create/update the events and taks, the other would always be in sync.

Unfortunately, the default file format used by Thunderbird ins’t .ics (or iCalendar format). By default, it stored the calendar in storage.db. But there is a way around it.

While Thunderbird does not use .ics file format for its own calendars by default, it supports use of iCalendar on the network (like Google Calendar) But Rainlendar calendar is not on the network.  So you trick Thunderbird (or lightning, depending on the way you look at it)

  • Go ahead, and create new calendar on network.
  • choose .ics format, but use file:// protocol to point to existing calendar created by Rainlendar.
  • Then you can delete the default one called Home. (If you have only once calendar, you are unable to delete it.)

Thanks to this thread on Ubuntu Forums for providing me this idea.

Reblog this post [with Zemanta]

Next Page »