Wednesday, 15 December 2010

QtCore.QDate gotcha for python dictionaries

I came across an odd little issue this morning while creating a data model for a diary I am writing.

In the model, Data is stored in a dictionary, with dates as keys.
For a variety of reasons, my first choice was to use a QDate for the keys, however this produced odd results.
The following example code should demonstrate.

>>> from PyQt4.QtCore import QDate
>>> mydict = {}
>>> for year in range(2009,2013):
...     dt = QDate(year,1,1)
...     while dt.year() == year:
...         mydict[d] = "boo"
...         dt = dt.addDays(1)
>>>
>>> mydict.keys()[:3]
3: [PyQt4.QtCore.QDate(2011, 8, 21),
 PyQt4.QtCore.QDate(2009, 12, 17),
 PyQt4.QtCore.QDate(2009, 7, 15)]
>>>
>>> mydict[QDate(2011,8,21)]
KeyError: PyQt4.QtCore.QDate(2011, 8, 21)

so what is going on? The dictionary DOES have a key of QDate(2011,8,21)... but is unable to find it.

In debugging, I was reminded that python dictionaries hash the key values and it turns out that there is a known bug in QDate.__hash__
>>> d1 = QDate(2009,12,9)
>>> d1.__hash__()
4: 48759696
>>>
>>> d2 = QDate(2009,12,9)
>>> d2.__hash__()
5: 45344152
>>>


this has been fixed in the very latest versions of PyQt4 I believe (including the python3 version)
In the meantime, I am simply adding a step of conversion to python date types for the keys.

>>> d1.toPyDate().__hash__()
6: 5917697570912074722
>>> d2.toPyDate().__hash__()
7: 5917697570912074722
>>> 

Thursday, 18 November 2010

a neat(?) postgresql solution using row_number() and self joining

For the scheduling aspect of openmolar, a common task is to find where "freeslots" occur. In the original openmolar, I performed this logic using python code. For openmolar2 the database itself will perform this search.
Here's how.

let's make a trivial diary table
create table diary (
 ix serial primary key, 
 startTime time, 
 endTime time, 
 activity varchar(80));
leading to this table of data.
select * from diary;
ixstarttimeendtimeactivity
109:00:0009:30:00breakfast
209:30:0010:00:00jogging
313:00:0014:00:00lunch
410:00:0011:30:00meeting with joe
514:00:0018:00:00golfing at Royal Dornoch

so the problem is this... how can I query that table to discover my free time on that day?
(there is 90 minutes before lunch)
NOTE - the activities are NOT in order!

the answer (and I found this in the excellent "sql cookbook") involves getting a view of that table in the correct order, and then performing a join on adjacent rows.

so first, get the ordered view.
=> select * from diary order by starttime;
ix starttime endtime activity
1 09:00:00 09:30:00 breakfast
2 09:30:00 10:00:00 jogging
4 13:00:00 14:00:00 lunch
3 10:00:00 11:30:00 meeting with joe
5 14:00:00 18:00:00 golfing at Royal Dornoch


But this doesn't help... because there is no way to definatively refer to the "next" activity, to do this we need to replace ix with a generated rownumber. We will use the row_number() window function of postgres 8.4.

=> select * from (select row_number() over (order by starttime) as rownumber,
starttime, endtime, activity from diary) as ordered_diary;

rownumber starttime endtime activity
1 09:00:00 09:30:00 breakfast
2 09:30:00 10:00:00 jogging
3 10:00:00 11:30:00 meeting with joe
4 13:00:00 14:00:00 lunch
5 14:00:00 18:00:00 golfing at Royal Dornoch

no we can self join that result set on itself..
(adding columns from the next row to the current row)

=> select * from
(select row_number() over (order by starttime) as row_number, * from diary) as diary1,
(select row_number() over (order by starttime) as row_number, * from diary) as diary2
where diary2.row_number = diary1.row_number+1

row_number ix starttime endtime activity row_number ix starttime endtime activity
1 1 09:00:00 09:30:00 breakfast 2 2 09:30:00 10:00:00 jogging
2 2 09:30:00 10:00:00 jogging 3 4 10:00:00 11:30:00 meeting with joe
3 4 10:00:00 11:30:00 meeting with joe 4 3 13:00:00 14:00:00 lunch
4 3 13:00:00 14:00:00 lunch 5 5 14:00:00 18:00:00 golfing at Royal Dornoch


or more succinctly...
=> select diary1.activity as first_activity, diary1.endtime as finishes,
diary2.activity as next_activity, diary2.starttime as starts from
(select row_number() over (order by starttime) as row_number, * from diary) as diary1,
(select row_number() over (order by starttime) as row_number, * from diary) as diary2
where diary2.row_number = diary1.row_number+1

first_activity finishes next_activity starts
breakfast 09:30:00 jogging 09:30:00
jogging 10:00:00 meeting with joe 10:00:00
meeting with joe 11:30:00 lunch 13:00:00
lunch 14:00:00 golfing at Royal Dornoch 14:00:00

so we simply now have to add a check to see if those times differ

so here's the final query!

=> select diary1.endtime as freetime_start, diary2.starttime as freetime_end from
(select row_number() over (order by starttime) as row_number, endtime from diary) as diary1,
(select row_number() over (order by starttime) as row_number, starttime from diary) as diary2
where diary2.row_number = diary1.row_number+1
and diary1.endtime < diary2.starttime




freetime_start freetime_end
11:30:00 13:00:00

Friday, 12 November 2010

Firesheep in Action

further to Yesterday's post on installing firesheep on 64-bit ubuntu .... I had a play with the firesheep plugin.

I disabled (temporarily) WPA on my home network, and started monitoring. I was able to hijack both my facebook and twitter sessions.

Thursday, 11 November 2010

Firesheep.

Firesheep is a plugin for Firefox that is creating a lot of noise in the IT security community. It allows you to hijack sessions of other users on open wifi.

There is a windows/mac version of this plugin, but that is no use to me (and I will point out my intentions are NOT malicious, but to demonstrate to friends/colleagues the dangers of the interwebs).

thanks to information on this page I got it working on my 64-bit ubuntu 10.04(lucid) laptop.

here's what I did to get it working.
regards

Neil.

to install firesheep on lucid

1. get dependencies (your mileage may vary)

sudo apt-get install libhal-dev libtool autoconf xulrunner-dev libboost-dev libpcap-dev iw git

2. get the latest firesheep code from github

git clone git://github.com/mickflemm/firesheep.git

3. compile the firesheep.xpi "plugin"

cd firesheep
./autogen.sh
git submodule update --init
make

4. set up a monitor interface
sudo iw wlan0 interface add mon0 type monitor
sudo ifconfig mon0 up

5. Install the plugin into firefox.
Open Firefox, and from the menu choose "File", then open ~/firesheep/build/firesheep.xpi

restart firefox when asked

6. firesheep needs correct permissions to access your wifi card.

cd ~/.mozilla/firefox/7oyiuecg.default/extension/firesheep@codebutler.com/platform/Linux_x86_64-gcc3/

sudo ./firesheep-backend --fix-permissions

note- there WILL be subtle differences in this directory

7. Ready to go!

open firefox and click on
add-ons->firesheep->preferences->interface
then choose mon0 from the drop down list. (see screenshot below)

8. Enable the Firesheep Sidebar.

view->SideBar->firesheep;
(or ctrl-shift-s)

DONE!

Tuesday, 28 September 2010

Object Orientation Example

I knocked this example up today for a friend on IRC. Thought I would post it here.
#! /usr/bin/env python
################################################################################
##  Python Object Orientated example by rowinggolfer                          ##
##  the same object... but extra functionality added in layers                ##
##                                                                            ##
##  A noteable "Gotcha" for newbs is the __init__ function.                   ##
##  the base class in this example sets some important attributes in this     ##
##  function. Therefore classes which inherit from it, but be careful if      ##
##  re-implementing this function                                             ##
##  NOTE - the teenager class does NOT overwrite this function... hence when  ##
##  a teenager is initiated, python looks up the class hierarchy and executes ##
##  the first __init__ method it finds.. in this case the Toddler's __init__  ##
##                                                                            ##
##                                                                            ##
##  Run this script, and study the output.                                    ##
##                                                                            ##
##  Pay special attention to the attribute "description, and see how it is    ##
##  altered mid method in the teenage years.                                  ##
##                                                                            ##
################################################################################

from datetime import date
from base64 import b64decode

class TimritBaby(object):
    '''
    a baby has a date of birth, a name, and limited functionality
    '''
    def __init__(self, date_of_birth):
        print "TimritBaby initiated"
        self.name = 'Scott Murtz'
        self.dob = date_of_birth
        self.description = "(a baby)"

    def poop(self):
        print "    %s %s can Poop!"% (self.name, self.description)


class TimritToddler(TimritBaby):
    '''
    a toddler has a new method
    '''
    def __init__(self, dob):
        # call the init method of the base class..
        # otherwise the name attribute won't work!!
        print "\nTimritToddler initiated"
        TimritBaby.__init__(self, dob)
        self.description = "(a toddler)"
    
    def walk(self):
        print "    %s %s can Walk!"% (self.name, self.description)

class TimritTeenager(TimritToddler):
    '''
    a teenager with a secret habit
    '''
    def private_time(self):
        self.description = "(now a teeneager)"
        message = b64decode('aXMgbWFzdHVyYmF0aW5nIGFnYWlu')
        print "    %s %s %s"% (self.name, self.description, message)


if __name__ == "__main__":
    baby = TimritBaby(date(1968,1,1))
    baby.poop()
    try:
        baby.walk()
    except AttributeError as e:
        print "ERROR - WHOOPS!", e
    
    teenager = TimritTeenager(date(1968,1,1))
    teenager.poop()
    teenager.walk()
    teenager.private_time()
    teenager.walk()
 

Sunday, 12 September 2010

fuzzymatching in postgres on ubuntu

I had a battle getting the soundex function installed into my database on postgres.
but here's how I succeeded.

One - Install the postgres-contrib package
sudo apt-get install  postgresql-contrib

Two - change user to postgres (a database superuser created on install of postgresql
neil@slim-maroon:~$ sudo su - postgres
postgres@slim-maroon:~$ 

Three - pipe the supplied script into the database (my database is called "openmolar_demo")
postgres@slim-maroon:~$ psql -d openmolar_demo -f /usr/share/postgresql/8.4/contrib/fuzzystrmatch.sql
SET
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
postgres@slim-maroon:~$

Four - test it!
openmolar_demo=> SELECT soundex('Neil');
 soundex 
---------
 N400
(1 row)

Tuesday, 7 September 2010

Postgres for openmolar2

I've been playing with postgres all morning, and have decided that the next version of openmolar will use it in preference to MySQL.

Main Reasons.
    1. it's a clean break with the past
    2. GNUmed use postgres
    3. active development
    4. standards compliance
    5. not owned by oracle
A huge thankyou to all postgres devs for a wonderful open-source back-end.

Thursday, 2 September 2010

Pynotify style notifications

The notifcation system for openmolar2 is nearly complete.

Why not use pynotify? Cross-platform is a goal here.

Wednesday, 4 August 2010

tllts co-incidence makes me feel awful.

so today, I am playing golf and listening to last week's tllts.
for the 1st time in my life, I get half way through the show (well, maybe 30 minutes in)... and i think "not enjoying this!" and delete it!

imagine what a douche i feel then, when I get home.... to find an air mail package containing a tllts T-shirt and hackey sack.

it's as if the cock just crowed 3 times, and i realise my denial.

Allan, Pat, Linc and Dann, please forgive me, it won't happen again.

openmolar2

last night I began work on a new version of openmolar.

this one will not be done in a hurry (ie. in 4 months because of a proprietary license termination), nor will I be bound to a schema which I dare not change. I have come a long way on my programming journey, and feel ready to take the plunge into what I hope will be the most customisable, pluggable, and downright froody piece of dental management software known to mankind.

openmolar up to this point, is software for my practice alone.
openmolar2 will be of use to other practices (I hope).

more later.

Neil.

Tuesday, 3 August 2010

QTextEdit with autocompletion using pyqt

Problem - I wanted a text-edit which would help enter long words.

Solution - I have converted the C++ example code into python, and post it here to for all to see.

NB - The code assumes your dictionary is located at /usr/share/dict/words.

from PyQt4 import QtGui, QtCore

STARTTEXT = ('This TextEdit provides autocompletions for words that have ' +
'more than 3 characters.\nYou can trigger autocompletion using %s\n\n'''% (
QtGui.QKeySequence("Ctrl+E").toString(QtGui.QKeySequence.NativeText)))

class DictionaryCompleter(QtGui.QCompleter):
    def __init__(self, parent=None):
        words = []
        try:
            f = open("/usr/share/dict/words","r")
            for word in f:
                words.append(word.strip())
            f.close()
        except IOError:
            print "dictionary not in anticipated location"
        QtGui.QCompleter.__init__(self, words, parent)

class CompletionTextEdit(QtGui.QTextEdit):
    def __init__(self, parent=None):
        super(CompletionTextEdit, self).__init__(parent)
        self.setMinimumWidth(400)
        self.setPlainText(STARTTEXT)
        self.completer = None
        self.moveCursor(QtGui.QTextCursor.End)

    def setCompleter(self, completer):
        if self.completer:
            self.disconnect(self.completer, 0, self, 0)
        if not completer:
            return

        completer.setWidget(self)
        completer.setCompletionMode(QtGui.QCompleter.PopupCompletion)
        completer.setCaseSensitivity(QtCore.Qt.CaseInsensitive)
        self.completer = completer
        self.connect(self.completer,
            QtCore.SIGNAL("activated(const QString&)"), self.insertCompletion)

    def insertCompletion(self, completion):
        tc = self.textCursor()
        extra = (completion.length() -
            self.completer.completionPrefix().length())
        tc.movePosition(QtGui.QTextCursor.Left)
        tc.movePosition(QtGui.QTextCursor.EndOfWord)
        tc.insertText(completion.right(extra))
        self.setTextCursor(tc)

    def textUnderCursor(self):
        tc = self.textCursor()
        tc.select(QtGui.QTextCursor.WordUnderCursor)
        return tc.selectedText()

    def focusInEvent(self, event):
        if self.completer:
            self.completer.setWidget(self);
        QtGui.QTextEdit.focusInEvent(self, event)

    def keyPressEvent(self, event):
        if self.completer and self.completer.popup().isVisible():
            if event.key() in (
            QtCore.Qt.Key_Enter,
            QtCore.Qt.Key_Return,
            QtCore.Qt.Key_Escape,
            QtCore.Qt.Key_Tab,
            QtCore.Qt.Key_Backtab):
                event.ignore()
                return

        ## has ctrl-E been pressed??
        isShortcut = (event.modifiers() == QtCore.Qt.ControlModifier and
                      event.key() == QtCore.Qt.Key_E)
        if (not self.completer or not isShortcut):
            QtGui.QTextEdit.keyPressEvent(self, event)

        ## ctrl or shift key on it's own??
        ctrlOrShift = event.modifiers() in (QtCore.Qt.ControlModifier ,
                QtCore.Qt.ShiftModifier)
        if ctrlOrShift and event.text().isEmpty():
            # ctrl or shift key on it's own
            return

        eow = QtCore.QString("~!@#$%^&*()_+{}|:\"<>?,./;'[]\\-=") #end of word

        hasModifier = ((event.modifiers() != QtCore.Qt.NoModifier) and
                        not ctrlOrShift)

        completionPrefix = self.textUnderCursor()

        if (not isShortcut and (hasModifier or event.text().isEmpty() or
        completionPrefix.length() < 3 or
        eow.contains(event.text().right(1)))):
            self.completer.popup().hide()
            return

        if (completionPrefix != self.completer.completionPrefix()):
            self.completer.setCompletionPrefix(completionPrefix)
            popup = self.completer.popup()
            popup.setCurrentIndex(
                self.completer.completionModel().index(0,0))

        cr = self.cursorRect()
        cr.setWidth(self.completer.popup().sizeHintForColumn(0)
            + self.completer.popup().verticalScrollBar().sizeHint().width())
        self.completer.complete(cr) ## popup it up!

if __name__ == "__main__":

    app = QtGui.QApplication([])
    completer = DictionaryCompleter()
    te = CompletionTextEdit()
    te.setCompleter(completer)
    te.show()
    app.exec_()

Wednesday, 30 June 2010

Session Managers and PyQt Applications

X11 window managers commonly have an option to "remember" applications which are running when a session ends.

Here's the gnome-checkbox (System -> Preferences -> Startup Applications)



but for this to work, the applications themselves need to comply. Qt does this with the QSessionManager class.

The class is not used directly, but is passed as an argument into functions of qApp, namely the saveState and commitData functions.
qApp therefore needs to be subclassed, and these two functions overwritten.

here's a working example. Note the simplicity of the functions!

#! /usr/bin/env python

from PyQt4 import QtGui, QtCore
import sys

class RestorableApplication(QtGui.QApplication):
    def __init__(self):
        super(RestorableApplication, self).__init__(sys.argv)

    def commitData(self, session):
        '''re-implement this method, called on quit'''
        pass

    def saveState(self, session):
        '''re-implement this method, called on run'''
        pass

app = RestorableApplication()

mw = QtGui.QMainWindow()
mw.setMinimumSize(300,300)
mw.setWindowTitle("I live on after a logout!")

label = QtGui.QLabel(
'''Leave me running and log off
I will survive! (on X11 systems)
Window size and position will be restored
... cool eh?''', mw)

label.setAlignment(QtCore.Qt.AlignCenter)

mw.setCentralWidget(label)
mw.show()

app.exec_()

Tuesday, 22 June 2010

qt-o-fax

Making the pluggable pyqt app has been more fun than I thought, and I believe the application may actually turn out to be half decent contacts manager.

it accepts plugins now (a zipped folder, with a config file within) and I am still finialising the plugin API, but I am certain I am on the right lines. The "zipimport" module did all that I hoped for and more.

It's running great on linux and windows. Plugins can simply be dropped into the applications plugins folder which reside here.
linux = ~/.qt-o-fax/plugins
windows = C:\Documents and Settings\USER\.qt-o-fax\
Config file is simple format, heavily influenced by the gedit method of plugin config
[qt-o-fax Plugin]
# module is where the entry point of the plugin is
# it should contain a class Plugin, with a method run()
# for this example, it is main.py
# THIS IS THE ONLY FIELD WHICH IS ABSOLUTELY REQUIRED
Module=main

restartneeded=False

Name=Hello World
Name[fr]=bonjour toulemande
Description=The simplest possible plugin
Description[fr]=Un plugin moins sofisticate

Version=0.1
Authors=Neil Wallace 
Copyright=Copyright © 2010 Neil Wallace
Licence=LGPLv3
Website=https://launchpad.net/qt-o-fax/

LongDescription=

Hello World Plugin

This plugin simply displays a hello world message when activated.
It connects to no signals, nor alters the database.

So what plugins could be created?? I think the sky is the limit. Add custom fields to the database, send SMS messages, create word processor documents with addresses embedded, christmas card lists.... contact sharing..

I've stuck it up on launchpad https://launchpad.net/qt-o-fax.


Screenshots




Wednesday, 16 June 2010

Making a pyqt4 application which accepts "plugins".

So I decided that openmolar needed to be broken up into bits. I did this last week, Client, Server and language packs are now seperate packages, meaning that I can update one part of it independently of the other bits. All well and good.

However, I REALLY want "plugins" (similar to firefox functionality) to provide some of the functionality that folks are going to want (partly because it will make it possible for them to code such things themselves)

So, as ever, 1st stage is to make a trivial app, and see if I can get some form of plugin system going.

Here's a video of that application (In making it, I learned a lot about QToolbar class... something I haven't used much before).
http://tinyvid.tv/show/2q6yruijgemsd
(if you have an HTML5 compatible browser, that should appear by magic below.. simply hit play)

If you want to see the code for this little application, I stuck it on launchpad - get it this way.

~$ bzr branch lp:~rowinggolfer/+junk/pluggable-pyqt-app

the vid was taken with the files at revision 1.

Now for the difficult bit.. designing and loading the plugins.

My idea is that plugins should be zipped folders. Signed by the author ideally, and containing a config file which describes it, gives version numbers, and installation instructions which the parent app can use.

Python can import modules from zipped folders easily thanks to the zipimport module.

I think I am on the right lines for this... I'll let you know (via this blog) how I get on


Friday, 7 May 2010

cloud required??

I have 2 issues to solve today.
1. backup of x-ray images (currently that is approx 3GB of data per year)
2. being able to share these images with colleagues when appropriate
now... forgive me if I am wrong.. but is the solution not "dropbox" or similar??

big opportunity for a lucrative market.


follow-up---
http://www.ironmountain.co.uk/solutions/industry/healthcare/imaging.asp