Continuous Integration and Deployment with Github

[CODE] ⇢ ⇢ ⇢ ⇢ ⇢ ⇢

Jenkins:

I didn't have good experience with Jenkins. In summary, something caused Jenkins to crash and I had to reinstall again... I repeated a few times and I decided that this should be done by someone else.
So I then went for hosted Jenkins. It was better but things weren't working everything perfectly just yet especially when creating test database.

I need super easy way to achieve Continuous Integration:

My co-worker mentioned that people at one conference was raving about codeship. Tested out free subscription. It is ridiculously simple and intuitive (just like when I found ansible while I was struggling with puppet). Best of all, there is small "?" button on bottom right corner and you can ask questions when you stuck and knowledgeable engineer would respond to you within 24 hours or so. Just fantastic.

Now to achieve Continuous Deployment:

What I end up doing is to setup deploy server which runs small flask (python micro framework) application and listens one type of request which is POST request from Github Webhooks.
Flask app will read payload from github and trigger ansible deployment script (via fabric) if commit / merge is made to master.
Once deployment is completed, it will post to private twitter account (this piece of code also on fabirc which is wrapper to ansible) that our team subscribes. The post message includes dev's commit message as well. I like this so far.
Because we are python shop and this approach will give us not only flexibility but ability to debug any issues easily.

python: advanced sort with sorted and lambda

I was working on Django project. I had need of order by QuerySet in somewhat complex way.
Basically, I am listing family but list "Spouse" relationship at the top.

Say I have data structure like this:

class Family:
    def __init__(self, type, name):
        self.type = type
        self.name = name

    def __repr__(self):
            return repr((self.type, self.name))

fam = [
    Family("child", "Joi"),
    Family("child", "Leona"),
    Family("parent", "Toshiko"),
    Family("spouse", "Coy"),
    Family("parent", "Akira"),
]

In SQL statement (PostgreSQL) I can express like this.

SELECT * FROM family WHERE ......

ORDER BY CASE WHEN type='Spouse' THEN 'aaa' else type END

Easy enough.
Now I wasn't sure how to do this in Django.
With raw sql or extras filter code becomes pgSQL specific solution so I wanted to avoid if possible.
So I decided to go with pure python function "sorted" since set of data I will be dealing with is small (just family members... so usually like 4 or 5 and at most... 10?)

Here is how I was able to accomplish.

>>> print(fam)
[('child', 'Joi'), ('child', 'Leona'), ('parent', 'Toshiko'), ('spouse', 'Coy'), ('parent', 'Akira')]

>>> family = sorted(fam, key=lambda x: ("aaa" if x.type == "spouse" else x.type)) 
>>> print(family)
[('spouse', 'Coy'), ('child', 'Joi'), ('child', 'Leona'), ('parent', 'Akira'), ('parent', 'Toshiko')] 

I don't probably need to explain what lambda or sorted with key param is as documentation and thousands of other posts are doing great job but I wanted to post this because I could not find example that shows this level of "key" for sorted function.
"key" will be return ('aaa') for Spouse and (<type>) for other relationship.

for x in family:
    key=lambda x: ("aaa" if x.type == "spouse" else x.type)
    print("%(name)s has type of %(type)s" % {"name": x.name, "type":key(x)})


>>> Coy has type of aaa
>>> Joi has type of child
>>> Leona has type of child
>>> Akira has type of parent
>>> Toshiko has type of parent

Ah-ha!

If you want to further specify sort order, you can add sort by name followed by type like this too.

family = sorted(fam, key=lambda x: ("aaa" if x.type == "spouse" else x.type, x.name))

Hope this magic became clear to you as well.

SonarQube to analyze python code on mac

SonarQube is an open platform to manage code quality. As such, it covers the 7 axes of code quality:
- architecture & design
- duplications
- unit tests
- complexity
- potential bugs
- coding rules
- comments


This is the minimum configured way to install SonarQube and start analyzing your python code on your mac (I run 10.8.5 but I think this should work on older os x as well)
Estimated time of completion: 10 min

1. Install SonarQube (server to host result)
1. go to http://www.sonarqube.org/downloads/ and click "Download" link and unarchived (I downloaded sonar-3.7)
2. move unarchived directory to wherever you want to place (I created dir called sonar. so far, i have sonar/sonar-3.7/ and I will reference this dir as SONARQUBE_HOME) 
3. cd SONARQUBE_HOME/bin/macosx-universal-64
4. to start server: $ sh sonar.sh start
1. at this point, if you start server (), you should be able to see interface via browser @ http://localhost:9000/
2. Install SonarQube Runner (server to host result) 
1. Download from here (at the time of writing 2.2 is the latest version) and unarchived it
2. move unarchived directory to wherever you want to place (I moved under sonar I created at step 1. I now have sonar/sonar-runner-2.3/ and I will reference this dir as SONAR_RUNNER_HOME)
3. Edit SONAR_RUNNER_HOME/conf/sonar-project.properties
• sonar.projectKey=my:project
• sonar.projectName=project
• sonar.projectVersion=1.0
• sonar.projectDescription=project description
• sonar.source=path-to-project-dir
• sonar.language=py
3.  Install python plugin
1. Download from here (at the time of writing 1.1 is the latest)
2. copy downloaded jar file to SONARQUBE_HOME/extensions/plugins/
4. Restart the SonarQube server
5. Run SonarQube Runner to analyze
1. note: I was getting "Java heap space" error. In my case, runner needed 269m. You can solve this by increasing limit and run as follows (example shows to set it to 1g but you can do 512m etc as well): SONAR_RUNNER_OPTS=-Xmx1024m bin/sonar-runner
2. now you can review your result @ http://localhost:9000/