Thursday 8 November 2012

Stack Overflow - it's hard work!

I've finally got around to creating an account at SO.
You can see how I'm doing here: http://stackoverflow.com/users/1351600/paul-c

The reason for doing this is to try and improve my own knowledge. If I don't know the answer to somebody's question then perhaps it's something I should know. So I can go research that and add the answer. And sometimes even get it right!

But it's interesting to find out how much you actually know about something rather then carrying on with what you think you know.

Some variation on "unknown unknowns" perhaps.

I've been quite surprised that sometimes where I thought I had a real understanding of something in fact I did not. Quite the opposite in some cases.

And of course the opposite is true too, I actually do understand some of this stuff quite well! At least, nobody's contradicted me yet and on SO that's not a rare occurrence.

What does astound me however is the slight variation on the "what have you tried" that I see all the time.

If you don't already know linking to "what-have-you-tried" is the rote answer given when somebody asks a question and essentially expects you do do their homework for them.

If you’re a developer and you’re about to ask another developer a technical question (on a forum, via email, on a chat channel, or in person), you’d better be ready to answer the question “What have you tried?”
This of course isn’t specific to software developers, but that’s my field and it’s thus the area in which I’m most familiar with the issue which motivated me to write this. I’m (sadly) quite sure that it applies to your own industry too, whatever that might be.
The thing is, there’s a disease in the software development world; a sort of sickness. It’s an unusual sickness in that it’s often not something you acquire once you actually join the industry (like greying hair, caffeine addiction and an ulcer), but rather it’s something that new recruits already have when they arrive.
It's well worth a read. Some people seem to have taken it to the next logical step however, of trying one thing, or a small number, and then just plonking the lot in somebody else's lap. That's it. I tried, I failed now please do my homework for me.

I dread to think of the number of people being sold development (as it can't all be hobby programmers) time that uses SO to solve basic problems.

So for example, there are some people that can obviously code on SO but they ask questions like:

The documentation says to do X = function(something, something_else) and I did that but it did not work. Please tell me why. 
Where of course the answer is going to lie somewhere else other then what is on the screen. Typically just stepping through the code line by line would answer many of these sorts of questions.

But if you look at some of the people they ask questions like this over and over again, questions that you'd expect they'd answer for themselves with just a little more effort then required to write the question.

So, what have they tried? They have tried a little. And now it's your turn. The best thing about SO is the score system. The worst thing about SO is the score system. The score system gives encouragement for these sorts of questions to be answered, so they get their answers. And I've answered a few myself.

But that's not what SO is about, to me anyway.




Monday 13 August 2012

Free Forum Software in google app engine.

I've been working on some simple forum software and it's almost ready to be released.

You can find a demo here: http://flowerace346663.appspot.com/forum/28002

The idea is that it attempts to run within the daily limits for app engine free accounts, so anybody can set one up without having to hire a server etc. And as long as your forum stays relatively quiet you won't have to pay a penny.

You can find quota details here: https://developers.google.com/appengine/docs/quotas

But it makes more sense to think about it for this software in terms of pageviews and posts. And I don't have those figures yet.

This will be free software for non-commercial usage with a parallel track for commercial usage or paid support and feature requests.

Programming side is mostly done, just the design left to go.

Friday 13 July 2012

The Search API - Part 1

The search API 

The Search API gives your application the capability to perform Google-like searches over structured data within your application. You can perform full-text searches across several different types of text (plain text, HTML, Atom, and others). Searches return a ranked list of matching text, and you can customize the ranking of results.
https://developers.google.com/appengine/docs/python/search/

Sound familiar? That "Google-like" search is really the main Google experience - ranked search results from a set corpus of data.

As usual Google provide excellent documentation, which I won't attempt to replicate here. Rather I'll talk about some of the problems I encountered when attempting an implementation.

The development Environment. 

Running the local instance of the development environment is very similar to running in the cloud. Similar but not identical. 

There are no resource limits locally, nor limits on the amount of operations per second. This can mislead as the cloud has very strict limits on some operations that are simply not present when running locally. 

My learning project was to take this book:


And create a front end for it to allow users to search the content and return relevant results.

You can see the front end here:


Yes, it's a "Dream interpreter". 

Luckily for me the layout of the book in it's electronic form was very amenable to processing into a set of separate documents. Each "dream" was formatted almost with this project in mind. 

_Thorns_.
To dream of thorns, is an omen of dissatisfaction, and evil will surround
every effort to advancement.
If the thorns are hidden beneath green foliage, you prosperity
will be interfered with by secret enemies.
Splitting the dreams up into separate list items was achieved with this regex:

 list_of_dreams=re.split('(_\w+\s*?.*?\w+_([\[][\d][\]])?)\.',data)

Then each dream interpretation was then further split into paragraphs with this regex:  '.\r\n'.

So for each heading (Thorns) we end up with one or more interpretations. Some dreams have dozens of interpretations so splitting then up allows a better search result to be returned.

So the ground is prepared - we now have a python list (in fact it's a little more complex then that) containing our target strings we want to search. Now what?

That's part two!





Subjects of interest

I'm currently learning every aspect of Google's cloud computing platform. Along the way I'll put together some tutorials and try to share what I've learnt. 

Google are now giving away all the technology that they use themselves to create Google! The exact same tools that they use internally are now available for you to use. 

Just look at what they have achieved with those tools! And now everybody can give it a go. 


The language I'll be using is Python.