Screen Shot 2016-05-18 at 13.54.49

StoryMap is a project that I worked on with Rift theatre company, Peter Thomas from Middlesex University and Angus Main, who is now at RCA, and Ben Koslowski who led the project. Oliver Smith took care of the tech side of things.  

The challenge was very specific, but the outcome was an interface that could work in a variety of public spaces.

We were looking to develop an artefact that could pull together all of the aspects of Rift’s Shakespeare in Shoreditch festival, including four plays in four separate locations over 10 days, the central hub venue where audiences arrived, and the Rude Mechanicals: a roving troupe of actors who put on impromptu plays around Hackney in the weeks leading up to the main event.

We wanted something in the hub venue which gave a sense of geography to proceedings. In the 2014 Shakespeare in Shoreditch festival the audience were encouraged to contribute to a book of 1000 plays (which the Rude Mechanicals used this year for their roving performances). We felt the 2016 version ought to include a way for the audience to contribute too.

The solution we ended up with was a digital/physical hybrid map, with some unusual affordances. We had a large table with a map of Hackney and surroundings (reimagined as an island) routed into the surface.

Screen Shot 2016-05-18 at 14.10.21

We projected a grid onto the table top. Each grid square could have a ‘story’ associated with it. Squares with stories appeared white. Some of the stories were from the Twitter feed of the Rude Mechanicals, so from day one the grid was partially populated. Some of them were added by the audience.

You could read the stories using a console. Two dials allowed users to move a red cursor square around the grid. When it was on a square with a story, that story would appear on a screen in the console.

Screen Shot 2016-05-18 at 14.18.52 Screen Shot 2016-05-18 at 14.18.10

If there was no story on the square, participants could add one. We had sheets of paper with prompts written on them, which you could feed into a typewriter and tap a response. Once you’d written your story, you put it in a slot in the console, and scanned it with the red button. (Example, Prompt: ‘Have you been on a memorable date in Hackney?’, Response: ‘I’m on one now!’)

Nearly 300 stories were submitted over 10 days.  Even though there really difficult to use, people loved the typewriters as an input method. Speaking from my own perspective, I found an input method that legitimised spelling mistakes and typos less intimidating. 

There were two modes of interaction – firstly, through the table based projection, which allowed a conversational, collective and discursive understanding of what had already been submitted.  Secondly, there was a more individual process of reading specific stories and adding your own story using the screen in the console. The second mode still relied on the projection, because you needed to move your cursor to find or submit a story.

The resolution of the projection was too low (because of the size of the table) for fonts or details to be rendered well. From this perspective, the map routed into the table really worked; it increased the ‘bandwidth’ of the information the table could convey, fine lines and small text worked well (which gave us a chance to play around with whimsically renaming bits of Hackney).

Having a way to convey spatialised data on a table where people can get round it and discuss it, combined with a (potentially private) way to add detail might work in a number of scenarios. Could it be a tool for planning consultation? A way to explore data spatialised in some other way, eg. a political spectrum or along a time line? Perhaps in a museum context?

The whole thing was developed as a web app, so it’s easy to extend across more screens, or perhaps to add mobile interaction. It’s opened my eyes to the fact that, despite all the noise around open data, there are relatively few ways to explore digital information in a collective, public way. The data is shared, but the exploration is always individual.  More to follow…

(I did a quick technical talk on how we delivered StoryMap for Meteor London, slides here.)

There were so many ways for Art Hackathon to go wrong, but more ways for it to go right than I realised too. Failure seemed so vivid in my mind’s eye, non-failure seemed so unlikely – at each step I couldn’t believe it all worked out.

Having vaguely committed to help Theo, Tom and Catherine put on a hackday about creativity and hardware (art?) I went on holiday for two weeks. I came back and discovered that tickets were going to be £20, and assumed this would be catastrophic or even fatal, but it wasn’t, and tickets sold. In fact they sold out. I was completely wrong to assume they had to be free, that was win number one.

Free because I knew we were going to have to promote it a lot, and as soon as people think you are making money they start mentally putting you in the spam category, which, I can say from experience, is incredibly disheartening. When I read this very touching blog about Hack Circus I instantly recalled the difficulties of promoting The Thing Is, a student magazine I helped run. We’d spend hours working to produce it, and then people would assign the most malign motives to us when we tried to get the word out. Forums (and hackspace mailing lists…) are incredibly hostile to people promoting things, even things that are highly relevant and not-for-profit.  Twitter, which didn’t exist when we did TTI, is fine with you promoting your projects. If you don’t like it, you can unfollow. Similarly, university internal mailing lists are very supportive.

For the record, we made no money and did not intend to. All of us, especially Theo and Tom, spent many many days on it.

Museum of Lies won the popular vote for best hack

Win number two was sponsorship. Theo got Ravensbourne Uni to sponsor us, effectively providing us with an amazing space for free. Unexpected lesson: open-plan office accoutrements are great for hacking. We were in the auditorium, but we were able to borrow big TV screens on wheels and also office dividers from around the Uni. Office dividers turned out to be great for making ad hoc structures for people’s hacks. Big TVs make hacking at scale possible.

Screen Shot 2015-05-19 at 00.37.21

Bare Conductive gave us conductive paint and it was a hit with hackers. Tessel were incredibly generous with us, gave us amazing hardware, and ended up hand delivering it from the US, because they are lovely (they were coming over anyway…)

Having seen these two bits go right, I started to worry that the dynamic on the day would be wrong. I imagined us finishing the talks, explaining all the hardware then saying “GO!” to the audience and them all just staring at each other, not knowing what to do. I got so paranoid about it that I caused an entirely pointless argument with Theo about the exact location of the chairs and tables, which I thought violated some kind of hacking feng shui, an entirely spurious concern.

We asked people to propose projects in our forum before the event, but very few people did. This only heightened my concerns. I should have had more faith, when we asked people to come to the front and pitch ideas about half of the participants did. There were too many ideas, not too few – fortunately teams were able to consolidate out of similar pitches and we ended up with a manageable number.

People at the front pitching ideas
People at the front pitching ideas

I can’t say if we could rely on that happening again, but it does make me think of a weird paradox in the way that I allocate time to the hackdays that I’ve been to. When I get emails from hack organisers I think “Don’t have time for this!”, and I never go and do whatever they want me to on their forum / google doc / IRC etc. Which makes absolutely no sense because I’m about to devote a whole weekend to the hack. In my mental accounting the hackday has to be boxed into a weekend timeline, otherwise I somehow feel it’s making an unreasonable demand on me. Perhaps other people feel like this.

And then at the end of it all people produced amazing hacks, hopefully we’ll have a proper video up soon. I wanted to use this space to record lessons learned, and the biggest one is that stepping out of your IT comfort zone is massively time consuming.

The winners (Scott Wooden and Chris Brown), who made a (highly addictive) web game, had an almost production ready app with animated transitions and beautiful graphics. Both of the team were using a language they use professionally (javascript), presumably using the tooling they use everyday at work. For them the hack was a chance to push what they already knew in a new direction, which they did very successfully.

Get The Banana, jury prize winner
Get The Banana, jury prize winner

Contrast that with a hack that starts with borrowing a Raspberry Pi from our hardware library. Even if you know Raspberry Pis a bit, there’s hours of flashing SD cards (if you want your preferred OS), finding out IP addresses, turning on SSH, discovering passwords before you can start. Wait, this is a Raspberry Pi 2? Does this library work with it? And so on…

I could do another post on the hostility of the Raspberry Pi as a platform, some of which I think is wilful, but there are two things I’d do differently if there was another chance and more resource. Two things other than sort out Raspberry Pi.

Firstly, I’d start out with hardware in functioning setups. Want a servo running off a Pi? Here’s one that we know works. Hack it if you want, but you can see it working now, so if it stops working you can probably work out why. If you really screw up, we could just flash you a new SD and rescue you.

Secondly, you can’t help teams very much when you are organising. I’d love to have spent more time helping hack, but I was too busy wrestling with an industrial scale coffee percolator or running the hardware library. There’s no solution to this except to have more people helping.

The hack was a laboratory too, we had two ethnographers looking at it, I was graphing the Twitter network around the event and there will be a follow up survey. Hopefully that will allow us to prove the value of the event to future sponsors, and also help us improve the next one, if anyone ever has enough energy to do one again.

A final lesson learned was the the output was so good that it was very sad to take it all apart after the show and tell – we could easily run an exhbition of the work which allowed more people to see what had been achieved. Next time…

 

 

 

 

 

 

 

 

 

I made a bookshelf at the weekend. As with the table that I built before (which I wish I’d written up), it involves no screws, nails or glue. I’ve tried to design it so that cutting and drilling isn’t required to be particularly accurate either. The idea with both is to put the complexity into the design rather than the build.

It took 10 standard, 8ft lengths of 2″x 2″ and about 3 meters of dowel. Each length of 2″x 2″ had to be cut once and drilled 3 times. Then I just threaded the dowel through to make a grid. To make it stand up, I tied string between the dowels on the back. Materials cost ~  £70.

You can concertina it back up if you wanted to – I don’t why you’d want that. Or take it right a part into its constituent parts.

Being “on the diagonal” means that you can use tension to make it stand up, unlike a standard “vertical / horizontal” bookshelf.  Well, actually, you could use tension on a “vertical / horizontal” bookshelf, but it would be hard to stop it wilting in one direction or another. On the diagonal, it balances itself.

The strings at the back are under quite a lot of tension, and each play a note when plucked. They are about C, A# and D, as determined using a guitar tuner. It might be possible to tune them properly by rebalancing the books.

If you look carefully, you’ll be able to see a VHS copy of Hangin’ With Leo. I hope Leo would appreciate the lengths I’ve gone to to store his video.

What do we really think about music? I’ve tried to find some data about how people think about musical genres using the Last FM API.

Ishkur’s strangely compelling guide to electronic music is a map of the relationships between various kinds of music, and a perfect example of the incredibly complex genre structures that music builds up around itself. He lists eighteen different sub-genres of Detroit techno including gloomcore, which I suspect isn’t for me. I wanted to try and create a similar musical map using data from Last FM.

I’ve written a bit before about the way in which the web might change the development of genres – what I didn’t ask was how important the concept of genre would continue to be. It’s difficult to listen to music in a shop, so having a really good system of classification means you have to listen to fewer tracks before you find something you like. Also, in a shop you have to put the CD in a section, so it can only have one genre attributed to it.

But on the web it’s easy to listen lots of 30 second samples of music, so arguably you don’t need to be so assiduous about categorisation. In addition, the fact that music doesn’t have to be physically located in any particular section of a shop also undermines the old system – one track can have two genres (or tags, in internet parlance).

Despite this online music shops like Beatport still separate music into finely differentiated categories, much as you would find in a bricks and mortar record shop. But do they reflect the way people actually think about their musical tastes?

Interestingly, two of the most commonly used tags on Last FM are “seen live” and “female vocalist” (yes, women have been defined as “the other” again), which aren’t traditional genres at all. “Seen live” is obviously personal, and “female singer” isn’t a part of the normal lexicon. Looking through people’s tags other anomalies crop up – “music that makes me cry” and tags based on where a person intends to listen to the music are examples.The more obscure genres from Iskur’s guide are lost in the noise of random tags that people have made for themselves. I would suggest Gloomcore isn’t used in a functional way that ‘metal’ or ‘pop’ are. It’s a classification that people do not naturally use to denote a particular kind of music on Last FM – perhaps it’s a useful term for writing about music, but nobody thinks they’d like to stick on some Gloomcore while they make breakfast.

I searched the Last FM database of top tags – the 5 tags most used by a user, and assumed that there was a link between any two genres that one person liked. For example, if you have ‘gothic’ and ‘industrial’ as top tags then I marked those two tags as linked. In the diagrams below I show the links that occurred between 1000 random Last FM users. If a link between two tags occurred more than about 15 times then it shows up on the diagram below.

Unsurprisingly, indie and rock are things that people often note they have seen live. By contrast, though people might talk of having heard electronic music ‘out’ (ie. not at home), they don’t care enough about it to use define a tag around it.

I was surprised to see tags such as ‘British’ and ‘German’, so I broke the above diagram down by country. Last FM has significant UK, German and Japanese user bases. Here is the result for Germany:

Media_httpjimmytideyc_cfhzk

I think it’s very telling that while most of the connections are as you might expect, ‘black metal’ and ‘death metal’ are not connected to the main graph. I’m not particularly aware of these genres, but it certainly seems plausible they are very insular.

Here is the Japanese version:

Media_httpjimmytideyc_ledsg

Yep, plenty of references to Japan. The only nation to feature Jazz too. Here is the British version:

Lost in the noise: what we really think about musical genres

In Japan and Germany a defining feature of music is that it is Japanese or German. In Britain we don’t care. I suspect that’s because our musical tastes aren’t defined against a background of lyrics in a foreign language, as perhaps they are in the other two countries.

Last FM may well have particular ‘subculture’ of user in each country, so its hard to draw any firm conclusions because of this potential skew. As with so many of the insights you can gain from data gleaned from the web, at the moment it’s only possible to tell that one day this kind of tool could be very reveling about our psychology – what it will reveal isn’t very clear yet.

None the less, it will be interesting to see how these diagrams evolve over time – perhaps they will gradually diverge from the old names we’ve used to identify music, or perhaps there will be less and less consensus about what genres are called.

Incidentally, this would have been a post about data from Linked In, looking at the way your professional affects the kind of friendship group you have, but the Linked In API is so restricted that I gave up.

The data is available blow. It’s in the .dot format that creates these not very sexy spider diagrams.

http://jimmytidey.co.uk/data/lastfm_genre_links/global.php
http://jimmytidey.co.uk/data/lastfm_genre_links/germany.php
http://jimmytidey.co.uk/data/lastfm_genre_links/japan.php
http://jimmytidey.co.uk/data/lastfm_genre_links/uk.php

I can provide a better version of this data if anyone wants it – send me a message.

Over the course of the General Election I recorded 1000 random tweets every hour and sent them to tweetsentiments.com for sentiment analysis.

Tweetsentiment have a service which gives one of three values to each tweet. ‘0’ means a negative sentiment (unhappy tweet), ‘2’ a neutral or undetermined sentiment and ‘4’ positive (happy tweet). Similar technology is used to detect levels of customer satisfaction at call centres by monitoring phone calls.

Obviously it’s difficult for a machine to detect the emotional meaning of a sentence, especially with the strange conventions used on Twitter. Despite this Tweetsentiment seems to be fairly reliable – tweets always which express happy emotions tend to be rated as such, and vice verse. More accurately, if Tweetsentiment does make a classification it tends to get it right. Sometimes an obviously positive / negative tweet gets a ‘2’, but that shouldn’t affect things here.

My hypothesis was that the Twitterati would be less happy if there was a Conservative victory. Of course I can’t prove that Twitter has a bias to the left, but I would presume that young, techy, early adopters are more likely to be left leaning. The reaction to the Jan Moir Stephen Gately article perhaps supports this.

David Cameron famously noted that Twitter is for twats, I wondered if Twitter would reciprocate…

Media_http1bpblogspot_uonky

The graph indicates that usually Twitter is just slightly positive, with a mood value of 2.1 on average. As predicted, as a conservative victory becomes apparent on Thursday evening there is a decline in mood which lasts until Saturday lunchtime. Then everyone cheers up, presumably goes down the pub, and is pretty chirpy for Sunday lunch. Sentiment only returns to average for the beginning of work on Monday morning.

In short, it does look like the election result was a disappointment to Twitter.

Obviously we need to know what normal Twitter behaviour is over the course of the week to draw very much information from the graph, and this is something that I’m going to try and produce a graph for soon.

It does look as though the size of negative reaction to a once-a-decade change in government is about the same magnitude as the positive mood elicited by the prospect of Sunday lunch – which I think is fairly consistent with the vicissitudes of Twitter as I experienced them.

I used Twitter’s API to gather the data, and frankly, it’s not particularly great, particularly if you want to get Tweets from the past. I was surprised to discover that any Tweets more than about 24 hours old simply disappear from the search function on Twitter.com – in effect they only exist in public for a day. For this reason the hourly sample size wasn’t always exactly 1000, but it was on average.

I’ll post again when I have some more data on normal behaviour. I’m also curious to find out if different countries have different average happiness levels on Twitter, but I think finding a Tweetsentiment-style service for other languages might prove difficult.

My last post used Wikipedia’s list of dates of births and deaths to build a timeline showing the lifespans of people who have pages on Wikipedia. There are a lot of people with Wikipedia pages, so I limited it to only include dead people.

That still leaves you with a lot of people to fit on one timeline, so I wanted to prioritise ‘important’ or ‘interesting’ people at the top and show only the most ‘important’ 1000. Some have been confused by my method for doing this, and others have questioned its validity, so this post will address both issues. I’m also going to suggest an improvement. It turns out that whatever I do Michael Jackson is more important than Jesus. I’m just the messenger.

Explaining the method
To get a measure of ‘importance’ I used work done by Stephan Dolan. He has developed a system for ranking Wikipedia pages which is very similar to the PageRank system which Google uses to prioritise its search results.

Wikipedia’s pages link to one another, and Stephan Dolan’s algorithm gives a measure of well linked to all the other Wikipedia pages a particular page is. If we want to know how well linked in the page about Charles Darwin is the algorithm examines every other page in Wikipedia and works out how many links you would have to follow to get from the page it is examining to the Charles Darwin page using the shortest route.

For example, to get from Aldous Huxley to Charles Darwin takes two links, one from Aldous to Thomas Henry Huxley (Aldous’s father) and then another to Darwin (TH Huxley famously defended evolution as a theory). Dolan’s method calculates the average number of clicks from every page in Wikipedia to the Charles Darwin page, and then takes an average value. To get to Charles Darwin takes an average 3.88 clicks from other Wikipedia pages.

Equivalently, Google shows pages that have many links pointing to them nearer the top in its search results.

This method works OK, but it could be better. For example Mircea Eliade ranks as the fifth most important dead person dead person on Wikipedia, taking on average 3.78 clicks to find him. But Mircea Eliade is a Romanian historian of religion – hardly a household name. We can take this as a positive statement, perhaps Mircea Eliade is a figure of hither to unrecognised importance and influence. On the other hand it seems impossible that he can be more ‘important’ than Darwin.

Testing the validity of the Dolan Index
I decided it would be interesting to compare what I’m going to call the Dolan index (the average number of clicks as described above) with two other metrics that could be construed as measuring the importance of a person. Before we do that, here is a Graph of what the Dolan index of dead people on Wikipedia looks like.

The bottom axis shows the rank order of pages, from Pope John Paul II, who is has the 275th highest Dolan index on Wikipedia, to Zi Pitcher, who comes 430900th in terms of Dolan index. It makes a very tidy log plot.

As I mentioned previously, the Dolan index is very similar to a Google PageRank, so lets compare them.

Media_http2bpblogspot_ydjxi

 

 

The x axis is the same as the first graph, Wikipedia pages from highest to lowest Dolan index. A well linked page has a low Dolan index, but a High PageRank, so I used the reciprocal of PageRank for the y axis. I’ve also added a log best fit line.

Comparing with PageRank seems to indicate there is a reasonable correlation between Dolan index and PageRank, which is indicated by the fact the first and second graphs have a similar shape.

PageRank is only given in integer values between 1-10 (realistically, all Wikipedia pages have a PageRank between 3-7), so I’ve smoothed the curve using a moving average.

This seems to lend some weight to the Dolan Index as a measure.

I’ve also made a comparison between the Dolan index the number of results returned when searching for a person’s name (without quotes) in Google search. It should be noted that this number seems to be quite unstable – a search will give a slightly different number of results from one day to the next. I’ve used a log scale because of the range of results.

Media_http2bpblogspot_djguv

 

There is barely any correlation here, except a very low values of Dolan index. Despite this, it’s still possible for the number of Google results to be useful, as becomes in apparent when trying to improve my measure of ‘importance’.

A suggestion for improvement
The problem with all the measures seems to be the noise inherent in the system. While Dolan Index, PageRank and number of Google results all provide a rough guide to ‘importance’ or ‘interest’ overall, each of them frequently gives unlikely results. How about using a mixture of all three? Here is a table comparing the top 25 dead people by Dolan index and using a hybrid measure of importance constructed from all three metrics.

Dolan index Hybrid measure
Pope John Paul II Michael Jackson
Michael Jackson Jesus
John F. Kennedy Ronald Reagan
Gerald Ford Jimi Hendrix
Mircea Eliade Abraham Lincoln
Peter Jennings Adolf Hitler
John Lennon Albert Einstein
Adolf Hitler William Shakespeare
Harry S. Truman Charles Darwin
Rold Reagan Oscar Wilde
J. R. R. Tolkien Woodrow Wilson
James Brown Isaac Newton
Anthony Burgess Elvis Presley
Elvis Presley Walt Disney
Christopher Reeve John Lennon
Susan Oliver George Washington
Franklin D. Roosevelt John F. Kennedy
Winston Churchill Timur
Ernest Hemingway Martin Luther
Theodore Roosevelt Voltaire

 

To get the hybrid measure I just messed around until things felt right. Here is the formula I came up with:

 

Hybrid measure = ((1/Dolan index)x 20) + (PageRank x0.6) + (log(number of results)x 0.6)

For some reason additive formulas give better results than multiplicative ones.

Using the hybrid measure seems to have removed the surprises (like Peter Jennings) although you might still argue that Oscar Wilde or Jimi Hendrix are much too high. Michael Jackson comes out as bigger than Jesus, but then he is an exceptionally famous person, and he died much more recently than Jesus. Timur (AKA Tamerlane) is a bit of a curiosity.

I considered ignoring Number of Google results because its such a noisy dataset, however it’s the only reason that Jesus appears at all in this list, he gets a very low ranking (4.01) from the Dolan Index. Any formula which brings Jesus out on top (which I think you could make a reasonable case for his deserving, at least over Michael Jackson!), gives all kinds of strage results elsewhere.

I am a bit suspicious of “number of google results” metric. In addition to volatility Number of results fails to take into account that occurrences of words such as “Newtonian” should probably count towards Newton’s ranking, but that people called David Mitchell will benefit artificially from the fact that at least two famous people share the name.

Any further investigation would have to consider what made a person ‘important’ – would it simply be how prominent they are in the minds of people (Michael Jackson and Jimi Hendrix) or would it reflect how influential they were (Charles Darwin for example, or the notably absent Karl Marx)?

I love the idea that the web reflects the collective conciousness, a kind of super-brain aggregation of human knowlege.

Just this week the idea of reflecting the whole of reality in one enormous computer systemwas promoted by Dirk Helbing, although my formula doesn’t rate him as very important, so I’m unsure as to how seriously to take this.

DBpedia mashup: the most important dead people according to Wikipedia

The timeline below shows the names of dead people and their lifespans, as retrieved from Wikipedia. They are arranged so that people nearer the top are the best linked in on Wikipedia, as measured by the average number of clicks it would take to get from any Wikipedia page to the page of the person in question.

I had imagined that Wikipedia ‘linkedin-ness’ would serve as a proxy for celebrity, which it kind of does – but only in a lose way.

Values range from 3.72 (at the top) to 4.04 (at the bottom). This means that if you were to navigate from a large number of Wikipedia pages, using only internal Wikipedia links, it would take you, on average, 3.72 clicks to get to Pope John Paul II. This data set was made by Stephan Dolan, who explains the concept better than me. Basically, it’s the 6 degrees of Kevin Bacon on Wikipedia.

I looped through the data set and queried DBpedia to see if the Wikipedia article was about a person, and if so retrieved their dates of birth and death.

The timeline does show a certain amnesia on the part of Wikipedia, Shakespeare and Newton are absent, while Romainian historian of religion Mircea Eliade comes 5th. If I had included people who are alive tennis players would have dominated the list (I don’t know why) – Billie Jean King is the second best-linked article on wikipedia, one ahead of the USA (the UK is number one!).

Any mistakes (I have seen some) are due to the sketchiness of the DBpedia data, though I can’t rule out having made some mistakes myself…

There results are limited to the top 1000, and they only go back to 1650. Almost no names previous to 1650 appeared, the exceptions being Jesus (who was still miles down) and Guy Fawkes.

In case you were wondering ‘Who’s Saul Bellow below?’, the answer is Rudolph Hess.

http://jimmytidey.co.uk/timeline/display/historical.php