bash-history - a tip and some thoughts.

A great tip on the Ubuntu-uk podcast this week for keeping your .bash_history file a little leaner (AND MORE SECURE!).

what's the difference between these two lines?

~mysqldump -u user -ppassword mydatabase > backup.sql
~ mysqldump -u user -ppassword mydatabase > backup.sql

well both do the same thing (backup a mysql database to a flat file).
However, the first variant would go into your .bash_history, whereas the second one wouldn't thanks to the space at the line start.

why does this matter?
1. you probably don't want to put plain text passwords into the shell history. It may be convenient, but this file is one of the first places anyone malicious (or simply mischievous) would look to harvest such things if they got physical or virtual access to your machine.

2. perhaps, like me, your .bash_history get's full of stuff like

cd ~
ls
cp -av /home/neil/bashpodder/2009-06-25/ .
mount /dev/sdb /media/player
sudo mount /dev/sdb /media/player
sudo -s
ssh -p 3298 -N -L 2948:192.168.88.2:3306 dentists@mywork.com
sudo umount /media/player
python
echo "http://feeds.conversationsnetwork.org/channel/itc" >> bashpodder/bp.conf
cd ~/windows_openmolar/openmolar/ && bzr pull

now of those commands, only one or two benefit from being available via arrow navigation. in future, I will be considering putting a space before the majority of my command line playing.

but d'you know what? I think I need to look into the behaviour of bash_history some more. Ideally I would like it to have everything from that terminal session directly available, spaces or no...(in case of typos, which I'll admit happen occasionally) but archive only those without.

I'll bet there's something in the man page... must check sometime.
Neil.

tunneling mysql over ssh

Here's my setup for remote access to my mysql server at work.

My work has a dynamically assigned ip from the local provider. I use the free service from dyndns.com to make that ip available. So let's say the hostname I chose from dyndns was mywork.dyn-alias.com.
I could ping that address using

~ping mywork.dyn-alias.com

I set up a machine at work (machine A) to listen for ssh connections on port 34567
so I can connect to that machine using.

~ssh -p 34567 mywork.dyn-alias.com

oh hang on.. my home user "neil" is not allowed, silly me, so let's say I have a known user called "dentist"

~ssh -p 34567 dentist@mywork.dyn-alias.com

that works!

Now A separate mysql server (machine B) resides on that network. It's LAN IP is 192.168.0.2.
This machine does not allow any other type of connection. What if I want to lever that database remotely? The easiest way is to ssh into machine A (as above) and use the mysql-client command line tool on that machine to connect to machine B.

that command is
~mysql -h 192.168.0.2 -u databaseUser -p

which will prompt for a password. or
~mysql -h 192.168.0.2 -u databaseUser -pPASSWORD mydatabase

which will pass the password "PASSWORD" automatically, and start using database "mydatabase" automatically.

but what if I want to use an application - like openmolar - to connect to that database.

the answer here is to forward the mysql port from machine B through machine A, so that it appears as a service running on my local machine.
here's how.....

~ssh -p 34567 -N -L 45678:192.168.0.2:3306 dentist@mywork.dyn-alias.com

which means...
-p 34567 use "port" 34567 (isn't port a silly term... surely this should be "channel")
-N from the ssh manual - "Do not execute a remote command" I don't fully understand this, but do know the connection is refused if I ommit this.

-L 45678:192.168.0.2:3306 let's break this down. -L is the command to "bind an address"

45678 is a random port on mylocal machine.

192.168.0.2:3306 is the address of the mysql service on machine B. machine A has permission to connect to this. Also note - 3306 is mysql's standard port.

with that command running, I can now point the app to 127.0.0.1:45678 and I'm golden.

dell mini bios upgrade

I've just upgraded the bios on my dell mini-9.

Followed this howto.
http://ubuntuforums.org/showpost.php?p=7336044&postcount=64

the new bios makes Fn+Z and Fn+X as function keys, and apprarently improves battery life.

more importantly... fancy graphics on boot :)

Floss Manuals

Fascinating podcasts have emerged from the libre planet event.

Many mention the work done (in sprints thereat) to improve the floss manuals.

Here they are. Highly recommended.
http://en.flossmanuals.net/

3 tips for new pythonistas

1. learn regex.
2. learn regex
3. learn regex.

Promote linux campaign

If I were involved in the marketing of Ubuntu... here is the advertising campaign I would run.

Setting - a school IT classroom
Music - Pink Floyd. "we don't need no education"
Theme - zombie kids being taught to use a popular word-processing application by (no names).

cool kid in the back of the classroom has a different desktop... what is it?
he's spinning the cube switching between all the killer apps. he winks to camera.

fade out...

ubuntu logo...

"because kids don't need no thought control"

openmolar update and a plea for sanity

2 weeks ago I switched off our commercially provided uber expensive crapware for the last time, and moved 100% to using my own solution "openMolar".

openMolar is a multi-client/single server application. Presently I have 2 linux clients, 3 windows clients running my software and a linux mysql server.
(I also have the full front-end and backend application running on my dell-mini9, but that's another story...)

The development of this application has been quite a trip, but proves that even someone with my limited coding skills can knock up a professional quality app using only open source tools.

And the bandwagon keeps on rolling.
Bryan Harris from the US and Mehmet nur Olcay from Turkey have joined in on the launchpad site, and are working on packaging the app for ubuntu and (more interestingly) Pardus linux. I look forward to making the code accessible for language translation purposes.

Richard Querin has again exceeded all expectations by providing an alternative logo focusing on the freedom aspect of the software. I intend to get some embroidered t-shirts for the practice staff to wear when we attend the British Dental Association Conference in 4 weeks time.


But before folks get too carried away, a reminder. This bandwagon needs the brakes applying somewhat.

My focus has always been (and will for the medium term at least) to have this software working in the practice.
The interest that is being shown from the dental profession and open source movement alike is quite humbling but a little frightening. It goes to show that there is demand for open source solutions, however, please let nobody be under the impression that this product is finished.
I am wary that if others take my code and push it out at this stage it could get it a reputation from which it will never recover. I am also fully aware that there are many companies who would love to see this project fail. I beg the open source community not to play a part in making this happen.

Here is a page of screenshots and explanations for those who would like to know more.

The project can be found on launchpad.

logos, launchpad and bizarre experiences

Three things.

1. I am overjoyed with Richard Querin's first draft of a logo for openMolar.
   


2. My code is up on launchpad.net at http://launchpad.net/openmolar.
   
3. The python versioning control system Bzr is working well. I hope. thanks to "thumper" in #launchpad on freenode for his guidance. I have made 3 pushes already. Awesome.
   

openmolar - new tarball

another test tarball for interested folks to install and try.

Still immature software, but a fair few new features.

Many thanks to all who sucessfully installed the 1st version despite a few errors in my README file!
Hopefully those bugs are now ironed out.

please note, you will need to refresh to the new database if you installed 0.0.3 version.

BTW - hopefully I will get the hang of bzr / launchpad for future releases... meantime, the tarball can be found HERE .

two wee python lifesavers

filed under the "notes to self" category... if I forget these tips I'll googling for them... so let's put them on my random blog for now.

• This should be enabled by default in the standard python shell. http://blog.venthur.de/2008/07/06/tab-completion-in-pythons-interactive-mode/


• 'enscript' is a must have for printing code... and from the man page, is this suggestion from the developer himself - flags for printing code: gaudy header, two columns, landscape, code highlighting, 2-up printing.

  ~$enscript -G2rE -U2 main.py

  
  if you have a long script, check it out... it's beautiful.
  

openMolar - it's not a comic book reader.

I was a guest on the linux link tech show last night, I was my usual incoherent babbling self, but I had a great time.

If anyone is interested in seeing just how badly I write code, please download,play with, and scrutinise the openMolar software.

tarball (inlcuding an example database for March and April 2009) and a checksum for the paranoid can be found at
www.invernessrowingclub.co.uk/misc/openmolar

Review - sansa clip ogg/mp3 player

With Linux cranks and the bad apples podcasts going to ogg only, I decided to invest in a new player.

The sansa clip seemed to fit the bill perfectly, for the following reasons.

• mounts as a drive, easy drag and drop of files


• plays ogg


• built in fm radio


• small


• cheap


• sexy

but it turns out that this is the best damn player I've ever owned... and here's why.

• It can delete tracks. Very useful. I need to do this, otherwise I forget what I've listened to.


• it can play tracks at an increased speed

That last feature sounds like a gimmick (or a discourtesy to the podcaster) however, I am enjoying it.
This morning, for example - it compressed lottalinuxlinks 106 down to MY commuting time, not Daves. Lynn didn't sound so sexy.. but kajarii did ;)

Battery Life could maybe be better... I have been spoilt by Sony players, having said that, battery life is perhaps the only thing to recommend Sony players.

dell mini ram trouble

Odd problem with my dell mini on Saturday. I needed to get a few packages up to ubuntu intrepid versions, and found that wasn't a trivial affair given the "lpia" archictecture that the pre-installed 'dellbuntu' insists upon.

I was installing dependencies, but they were still appearing to be missing. Very odd, so I decide to install the vanilla 32 bit Ubuntu8.10.

I partition the drive (ext2 with no swap) and stick in an 8.10 usb startup disk. Start the install and BOOM!! major problems at 27%. I get warnings that "source and destination files don't match".... "you probably have a dirty CD, or a damaged hard-drive".

Bummer - it has to be the hard-drive right? We all know these solid state drives are friable?

A quick google suggests several folks have had trouble with the dell mini drives, so I was 100% certain that was the problem.
Anyway... long story short, it turns out it wasn't that at all.

My RAM was hosed. memtest86 on the startup disk (which I tried on a whim whilst googling for SSD device checking software) found 40,000 errors at a very specific range of addresses.

So I popped in a different ram block, and voila! All is well. Ubuntu 8.10 was running flawlessly 20 minutes later.

On the off chance any ubuntu devs reading this.. it might be worth adding a line to the error dialog stating that the memory could be an issue as well as the more likely CD or hardrive cause.

Even more importantly, THANKYOU for having that memtest86 on the install media by default.. you saved me from ringing dell support (a fate worse than death methinks?)

Using binary data in python

For those who don't know, I am writing a python gui-frontend to an existing mysql database.

The original program that wrote out the data was written by an old-school C guy. Some of the data has been put into the tables in a "blob" format. Example

mysql> describe esttable;
+----------+-------------+------+-----+---------+-------+
| Field    | Type        | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+
| serialno | int(11)     | NO   | PRI | 0       |       | 
| data     | blob        | YES  |     | NULL    |       | 
+----------+-------------+------+-----+---------+-------+

so when I read this table with the MySQLdb module (a third party module for python) I get a string like this.

import MySQLdb
db=MySQLdb.connect(host="localhost",user="myUser",passwd="myPassword",db="myDatabase")
Cursor=db.cursor()
Cursor.execute('select data from esttable where serialno=31599')
Cursor.fetchall()
(('\x01\x00o\x00\n\x0f\x00\x00\x04\x00\xc9\x00\x00\x00\x00\x00\x01\x00
\xe9\x03"\x0b\x00\x00\x02\x00\x87\x05@\x1f\x00\x00',),)

You will note that this is packed with characters which need to be handled with care (understatement!)

So how to make sense of it?
One way is to use python's build in Struct module, which allows conversion between python and C types of variable.

after a bit of decoding.... I noticed the above is in chunks of 8 bytes, with /x00 at every 2,6,7 position. I also realise (by experiment) that the data contains 2 distinct unsigned doubles so the following code works for me.

import struct
def blobToList(blob):
   pythonlist=[]
   i=0
   for i in range(0,len(blob),8):
       number=struct.unpack_from('b',blob,i)[0]
       item=struct.unpack_from('H',blob,i+2)[0]
       cost=struct.unpack_from('H',blob,i+4)[0]
       pythonlist.append("%s %s $%d.%02d"%(number,item,cost/100,cost%100)) 
   return pythonlist
if __name__=="__main__":
    testdata ='\x01\x00o\x00\n\x0f\x00\x00\x04\x00\xc9\x00\x00\x00\x00\x00'
    testdata+='\x01\x00\xe9\x03"\x0b\x00\x00\x02\x00\x87\x05@\x1f\x00\x00'
    print blobToList(testdata)

which gives me the following output

['1 111 $38.50', '4 201 $0.00', '1 1001 $28.50', '2 1415 $80.00']

and that makes more sense to me (just).

Wikipedia for young minds

Another random idea inspired by real life.

Assuming there isn't (see picture)... wouldn't it be nice if there was a version of wikipedia aimed at kids under 12.
Same as wikipedia (and linking to it), but less verbose, simplified language, and guaranteed free of adult material?

Heck that would be a great resource for junior schools, teachers and parents alike.

Possible Names - "WikiJunior", "Junior Wikipedia", "PaedoPedia" (ok... one of the 1st 2)