I’m moving my blog
Check out http://phalt.co/article from now on for all my blog posts.
Stopspam has achieved over 1,000 downloads already in Alpha version.
This is pretty remarkable for something I built in my spare time and I’ve wanted to improve the package ever since.
Because of this great response, I decided to continue building stopspam and make some huge changes for the first Beta version - 0.4
The major changes are:
* New API functionality to simplify things.
This was done to make stopspam explicit and simple to use, older functions are still present but have deprecation warnings on them.
* Batch mode
A new ‘batch mode’ function to pass a list of items to check them all together. This is a far more efficient way to use the application as it reduces API calls through to the stopforumspam database.
* Readthedocs integration
If this is to be a successful open source project it requires correct documentation. Readthedocs documentation has been generated and will be updated with each new future release of stopspam. Read the docs here.
* Travis CI integration
Travis CI provides automatic testing with each new build that is pushed to github. This is a very efficient way of testing builds and they also provide nifty widgets to put straight onto the README on the Github page. This’ll help other potential contributors see the state of the project and what needs to be done.
* Removed terminal functionality
The new API functionality is much easier and quicker to use than the terminal mode, so it was removed. It is pointless writing code twice - remember kids: Don’t Repeat Yourself!
and with that, we have version 0.4, the first Beta version.
Download it through pip easily:
pip install stopspam
Looking ahead - making stopspam better
The original purpose of stopspam was to make my life easier when detecting spam on a few websites I manage, now I’ve realised it has a lot more potential than that. A lot of spam filters are a constant annoyance for developers and website administrators. Often they can be too paranoid and stop everything on the website and just as often they can be too lax and let everything through.
Anti-spam tools need to be fixed, that’s what stopspam aims to do.
By integrating different methods of detecting spam, stopspam can potentially become a reliable and easy to use method of detecting and stopping spam.
It isn’t enough to use automatic filters that process words in a string and figure out if it is spam or not, because that often leads to false positives and it causes issues with the genuine users on a service.
It isn’t enough to rely on a database to determine if something is spam or not.
A library that uses a mix of both services with its own unique methods of processing suspects and their potential as spam would provide the best outcomes.
I’m going to make this my open source project to work on and I’d love support - if you feel like you’re fed up with bad spam filters, this is the chance to help build and develop the future of anti-spam software.
Coming soon is version 0.5 - better caching and hopefully some django integration too!
bhd2 asked: For DjangoPi install what should the install directory be?
DjangoPi will create a virtualenv folder called ‘djangopi’ where ever you run the setup script.
Virtualenv is a tool for creating ‘sandboxed’ development enviroments, it will install Django and a host of other things within that sandbox. For more information on Virtualenv see http://www.virtualenv.org/en/latest/
Today I released my first open source contribution to the Python Package Index.
Stopspam is a python library for discovering if an ip address, email or username is likely to originate from a malicious source.
Stopspam wraps up the api from stopforumspam.com, a website I’ve used in the past to check for malicious user accounts.
Install and use
Stopspam can be easily downloaded with pip:
pip install stopspam
and then you can start discovering spam:
Stopspam also includes a way to check the confidence level of an ip address, email or username. A higher number (out of 100) means they’re more likely to be malicious:
If you want to get back the data directly from stopforumspam.com you can use the check_* functionality built into stopspam:
d = stopspam.check_ip('127.0.0.1','json')
data = json.loads(d)
I built stopspam because I was fed up with having to manually check thousands of ip addresses and usernames against a database at stopforumspam.com. I also realised that there wasn’t anything generic available for Python, Django, flask or any other python-powered web frameworks.
I designed stopspam to be stupidly simple on purpose - it doesn’t try to do anything tricky or confusing. This way developers can incorporate it into their own solutions easily.
Suggestions for use
Stopspam is very minimal and alone will only provide information on malicious ip addresses, emails and usernames. Developers will need to use stopspam to contribute to their current anti-spam solution.
I’d suggest using
check_*_confidence() to fully determine if a user account or ip address is in fact malicious, such as in the code below:
import stopspam as spam
if spam.is_spam(ip) and spam.check_ip_confidence(ip) > 90:
# It's probably spam, so get rid of it
I really want to improve stopspam and create some hooks for it so it can be used well in Django - maintaining a list of users or visitors who could potentially be malicious would be stupidly helpful.
I’d also like to create a batch checker that runs through a list or a file of ip addresses, emails and usernames and returns all the ones that are likely to be spam.
I want to see people use stopspam! Download it, incorporate it into your projects and contribute to the code on github!
If you’ve used stopspam to help combat spam on your website, then let me know on twitter
This is part of a longer series of blogs about research I am conducting for a module at university.
As I mentioned in the last blog, I created a survey to get some feedback from users on how they use Facebook and how they think it effects their mood.
I deliberately chose to conduct this survey before doing any research, so I can examine the results, evaluate the information and discover what I need to do right in order to conduct a proper survey that yields more valid results.
Obviously, it could be that this survey does yield interesting results, but I wasn’t hopeful.
I sent the survey out on Facebook, various Facebook groups, Twitter, via email and also to a few teachers, who passed it on to their students.
I ended up with 78 participants, a much larger number than I originally believed to get. Hopefully I’ll be able to get this big a result next time too.
How often do you use Facebook?
This seemed like an obvious place to start and I got back an answer I was expecting - 51% of users check it daily, whilst a further 24% check it hourly.
Why is this useful? I believe the amount of time a user spends on Facebook will have an effect on their happiness - a larger exposure could have a greater effect. I’ll use it in tandem with other statistics later on to test this.
How many friends do you have on Facebook?
This again, seemed like an obvious question to ask.
62% of users had over 200 friends.
This, like the previous question, is related to the possible amount of exposure a user might get - do more friends equal a more positive experience on Facebook?
Why do you use Facebook?
Another obvious question to ask:
With some obvious results - 87% of users talk to friends, whilst on 44% talk to family.
There is also an interesting comparison - 76% use it to see what their friends have been up to, whilst 42% use it to post about what they have been up to. This suggests that most users consume information, rather than contribute to it. In the case of Facebook - the information they’re consuming is their friend’s activities.
In terms of this research this is interesting - because I want to see how Facebook effects a user’s mood, these results tell me that people mostly care about what others are doing. Does this mean it’ll have a major impact in said mood? If the user cares about others being happy, will it ‘rub off’ on them too?
Do you get excited to check Facebook?
An interesting question, I guess I wanted to see if people felt an ‘addiction’ to checking Facebook. The results were unsurprising:
Only 7 people said they got excited to check Facebook. I decided to delve into this more and discovered that every person who got excited to check Facebook also said they checked Facebook hourly. An interesting result - are these the few who feel a satisfaction from checking Facebook?
The mood of my statuses / the mood of others
These two questions were asked about the ‘general attitude’ of statues - your own and others on your feed.
I’ve paired them together because they’re interesting:
The results from this survey suggest people find their own statuses far more positive than that of others. Even though, as we saw earlier, most people used Facebook to see what others were posting and not to post.
The results couldn’t be more perfect - 47% said other statuses were neutral and 47% said their only statuses were mostly positive!
Is there a bias here? I’m pretty certain of it, we perceive ourselves differently to how we perceive others because we only base on others what they show - in this case, their Facebook statuses.
A generally happy person who posts nothing on Facebook except for a single negative status or comment will give off a biased vibe about their mood. On the flip side - a negative person could post happy things and give the exact opposite vibe.
What mood do you usually have AFTER checking Facebook?
I left this question to the end after they had hopefully looked over their feed and actually checked their Facebook.
Surprisingly - most people didn’t feel like their mood had changed at all.
I delved into the results more and discovered that the people posting the positive answers at the top checked Facebook more often than those who put negative answers.
This again can be a biased question - their mood during the time of the questionnaire can have an effect. Also - users may not consciously be aware of their mood changing at all during use, especially those who use it often. In fact - most users who use Facebook hourly said they had no noticeable change to their mood.
Now that I have had some results from a survey, I will continue to read and research papers related to group happiness and wellness.
I’m also interested to look into the psychology behind the ‘like’ button on Facebook - often used as a quick, gesture-less way to give someone a compliment. There is also the interesting fact that there is no ‘dislike’ button, is this a deliberate design decision?
Let’s hope Congress does not flinch as it begins the debate about immigration reform because the future is passing through security – in the wrong direction. It leaves the United States on every departing airplane carrying a foreign born student who has graduated from an American university with an advanced degree in the sciences, technology, engineering and math. The majority of these people want to stay in the United States but because of existing immigration laws, they have no choice but to leave.
In Silicon Valley, which has always been blind to any attribute other than ability, everyone knows that the remarkable achievements of the foreign born have led to the formation of companies such as Google, Intel, Sun Microsystems, nVidia, Yahoo! PayPal and scores of others that are less well known. Of the last eleven early stage companies that have allied themselves with Sequoia Capital, seven have had immigrants among their founding lineup. This is not a sudden or recent phenomenon; it has been the leitmotif of our business since the 1970s. However, the number of startups would be even higher if we weren’t ejecting foreign-born students and if we welcomed their contemporaries who have been educated overseas. Today, it is impossible to satisfy Silicon Valley’s appetite for engineers and scientists with people born in America.
This is part of a longer series of blogs about research I am conducting for a module at University.
"Does Facebook effect our mood? An investigation into how the content we read everyday on Facebook effects our emotions and our general outlook on the experience of life."
This is the working title of the paper I intend to write for this module.
For Advanced Web Technologies we are:
required to carry out a practical investigation into the following topic: “Information management and delivery in web scenarios.” This can be based around any web technology.
Whilst quite vague, it actually opens up a multitude of opportunities. One of which I have already spent some time being interested in: How social networks influence our emotions.
I have always wondered how one’s mood might be if they spent all day doing nothing and then saw a bunch of their friend’s posts about what a fantastic day they had, or seeing a bunch of holiday pictures uploaded.
Would it inspire them to go out and enjoy life?
Would they be happy seeing their friends happy?
Would they get extremely jealous?
Would they end up depressed because they’ve wasted the day?
This is a subject I believe I have a lot to talk about and a lot to investigate into. I also have a prime source of data just waiting for me - my friends on Facebook, and my direct access to their statuses would give me some insight. A lot of my friends are also very co-operative, so I know that posting surveys will yield some useful results.
But first, I must make sure that my research will hit certain learning criteria for this module:
Demonstrate the application of evaluative methods to web technologies
Okay so, demonstrate my ability to research and collect data, then evaluate them? Brilliant! I’m doing that later on in this blog post.
Understand the ethical implications of solutions using web technologies
Ah, this might be an easy mark to hit as I’d like to cover the ethics around Facebook’s effect on our emotions.
There are of course, other criteria that must be hit or considered, but for now these are the most relevant (at this stage).
What research have I done already?
As I’m madly enthusiastic about this module, I started on my research almost straight away.
I used Google and the Uni’s online journal records to find a few journals covering similar fields.
One, Too Much of a good thing? The Relationship between number of friends and interpersonal relationships on Facebook by Stephanie Tom Tong, Brandon Van Der Heide and Lindsey Langwell investigated the effect of having a lot of friends on Facebook. It raised a few interesting, if barely relevant points:
Wall postings alluding to sociable behavior by the target increased favorable ratings of targets, whereas postings sug- gesting excessive drinking and philandering prompted a reversal.
Sociometrically popular individuals receive more positive ratings on measures of liking and potential friendship from peers
Some interesting points about our perception of people’s lives, but does that really effect our mood?
I watched a TED Talk, an 18 minute video presentation by Nicholas Christakis (http://www.ted.com/talks/nicholas_christakis_the_hidden_influence_of_social_networks.html) which looked at The Hidden Influence of Social Networks.
Whilst looking at all sorts of social networks, not just online ones, he discovered that obese people were more likely to influence their friends to be obese, or were more likely to have obese friends.
Whilst watching I asked myself - do emotions have the same effect as this?
Luckily for my, Nicholas started to talk about that exact thing:
Could emotion spread, in a more sustained way than riots, across time and involve large numbers of people, not just this pair of individuals smiling at each other in the subway car? Maybe there’s a kind of below the surface, quiet riot that animates us all the time. Maybe there are emotional stampedes that ripple through social networks. Maybe, in fact, emotions have a collective existence, not just an individual existence.
The talk is very interesting and I will be citing it often during my further research.
I have decided to create a survey for people to fill out that will ask how people feel about using social networks. I’ll ask the following questions:
I’ll also gather information like:
This will help me make a comparison between the different types of Facebook users.
From there, I plan to analyse the data before conducting more research and solidifying my next steps.
Today I realised that I no longer had a portfolio website.
I had a few sites like djangopi and my github account running with a bunch of my stuff, but there was no personal “about me” website on the internet that I could show off my skills and be all big-headed on.
So I decided I’d try and see how quickly I could produce a clean, beautiful website that showed off my design skills and also a few of the projects I had worked on in the past.
Right, I’m making a portfolio site in one day. Starting now. Here we go.— Paul Hallett (@phalt_) January 23, 2013
It took me about 4 hours.
Have a look at it here:
Here is how I did it:
It also has a really nice editor, though it did freeze and crash on me about 3 times.
Above is an in-progress shot of the jetstrap interface and how I was using it to develop my site.
You’ll notice that I had to write in things I wanted to do with the template afterwards - this is because Jetstrap, whilst awesome, is very limited in how much you can edit.
After getting a template of my website downloaded, I decided to look into web hosting and domain names.
I used namecheap to buy a cheap domain: phalt.co (I wanted the cool .co address) for around $20 a year.
A brilliant friend of mine, Robert Putt, runs a web hosting service and agreed to give me the cheapest possible web hosting he could offer. I picked the cheapest because 1. I’m a student and 2. I don’t expect the website to get many hits.
Configuring nginx took a while, but I eventually got the hang of it.
With the server set up and ready I jumped back to the website.
As this is just a static page, the longest part of the development was filling out the content, which meant grabbing images and writing text for an hour or so.
Once I’d got my website full of content and the styling finally tweaked I dropped it into a git repo and used github to store it online.
In fact, you can see the public code of my website here.
Finally, all that was left was to SSH back into my server and pull down the git repo, make some final tweaks to nginx and boom, it was live!
Later on I decided to add an email server so that I could get emails though firstname.lastname@example.org
So, what did I learn from all this?
And what am I going to do now?
For the past few weeks I’ve been taking classes of 15-17 year old students and teaching them a variety of programming concepts and practical knowledge in Python.
Until recently I had never taught anyone anything about these things, so the experience has been entirely new to me and I’d like to share a few things I’ve learned along the way.
Some are surprising to me, but might not be to you:
All of this has taught me one thing:
The Raspberry Pi really has helped me to start a decent education or interest in computer science with a lot of students.
I’d say in the past few months, I’ve taught around 80 students already. From the feedback I’ve gotten from teachers - about a third of them have pursued things further afterwards without any help from myself.
What is the next step?
After that, who knows? More schools, bigger classes, world domination?
A better educated country?
One thing is for sure - I believe educators and Raspberry Pi need to work together to help organise a lesson structure or guide on what can be covered with the Raspberry Pi.
I also thoroughly believe teachers need their own refresh or introduction course to cover these things. I’ve held a teacher training day with 10 teachers in October and they, like the students, absolutely loved this new style of hands on teaching.