15th Discovery Challenge

The challenge takes place at September 27th. Please find a detailed list of the schedule and a link to all accepted papers at our workshop section.

As the offline challenge has ended and the online challenge not yet started, we had a short look at your submitted result files. Though we only saw what was recommended, we already know that completely different approaches for the name recommendation task were applied. So we are very much looking forward to reading your papers and getting an impression of all the different pre processing and recommendation systems.

After passing the reviewing process, all papers of participants of the challenge’s workshop will be published in the workshop proceedings (for details, have a look at the workshop description). The workshop will take place on September 27th, that is, on the last conference day.

The introduction of the workshop proceedings will contain an introduction to the overall setup of the challenge as well as a detailed task description. You can therefore skip these details in your paper and just refer to the introduction.

If you have further questions concerning your workshop submission, please don’t hesitate to contact us.

Happy letter typing!

Once again we have some nice results to share! Today we look at Twitter’s ReTweet graph, based on the same data set which was described in Teaser #3. The ReTweet graph was extracted from Jure Leskovec’s Twitter Sample, applying a simple RT @username filter (thereby ignoring “dark retweets”).

The resulting ReTweet graph comprises 826,104 users with 2,286,416 edges. Just considering the ReTweet frequencies (i.e., how often user A retweeted user B), we aggregated some average similarity scores. Firstly, we collected all hash tags for each user separately and represented the user by the resulting hash tag context vector (i.e. each component of a user’s context vector contains the number of tweets in which the user applied the corresponding hash tag). Thus we can calculated the cosine similarity between pairs of users, based on the corresponding context vectors. Averaging these similarity scores per retweet frequency, we obtained the following plot (excluding self retweets):

As we can see: Pairs of users who retweet one the other more frequently, tend to be more similar with respect to the corresponding hash tag usage. This is not surprising, but nevertheless, nice to see. Please note that the plots are log-log scaled and retweet freqencies are binned logarithmically.

Secondly, we extracted geo locations for Twitter users and calculated the average geographic distance of user pairs, relative to the corresponding retweet count:

These results are not as clear as in the case of hash tag similarity, but nevertheless, we can observe the tendency of user pairs with higher retweet counts being more closely located. It is worth noting that the global average geographic distance of all users is 7,484 Kilometres and thus already low retweet frequencies yield significantly lower average distances. For your convenience, we also show the linear scale plot:

But finally, the interesting part: Again, we heuristically determined given names for Twitter users by matching the user name with our list of known names. We thus collected names for 179,260 users, having 111,204 links in the ReTweet graph (excluding self retweets). We than calculated the average name similarity of user pairs based on the name co-occurrence graph derived from the English Wikipedia corpus (as described in the Nameling papers):

The result is rather unexpected: The average name similarity decreases with increasing retweet counts! That is, spontaneous retweets are more likely among users with similar names and user pairs which retweet often tend to have less similar names.

At this point, further investigation is due. Maybe these results are artefacts induced by the applied name similarity function. But other hypothesis may also support these observations. Higher average name similarity for low retweet counts can be explained by assuming that spontaneous retweets are more likely related to topics which are relevant to the retweeting user’s cultural background (e.g. local events, TV shows, etc.) and for user pairs who retweet often, the name correlated relations are less important, as these users share some focused interest (e.g. Recommender Systems).

These are of course only speculations and we welcome you to discuss these observations either via Twitter or in our forum!

Happy number crunching!

Once again, it’s time for some number crunching fun… Today, I looked at the interrelationship of first names within Twitter’s Follower graph and got some beautiful results.

For the analysis, I used an excerpt of the Follower graph, consisting of 1,486,403 users and 72,590,619 links (as described here), as well as the name co-occurrence graph based on the English Wikipedia corpus which is used for calculating name similarities in Nameling (as described in the Nameling papers). The fist names of Twitter users were extracted from the users’ profile data, where a user may provide her or his full name. Of course, many users just entered some fantasy name. Accordingly, the first token of the provided name string which matched against our list of known names was chosen as the user’s first name. This process induces some noise into the data, but due to the vast number of considered pairs of users, this effect should be neglectable.

Now, relative to 3,078 randomly chosen users (our Linux cluster is still crunching on more), I calculated the average name similarity of direct neighbours in the Follower graph, the average name similarity between pairs of users at a (shortest path) distance of two, …of three, and so on. For reference, I also added the total average name similarity for all considered pairs of users, as depicted by the grey dashed line. Finally, the error bars correspond to the 95% confidence interval.

As we can see, users which are located more closely within the follower graph tend to have more similar names than distant users. Additionally, a monotonically decreasing dependency between the average name similarity and the shortest path distance in the follower graph can be observed. Moreover, users at a distance up to three tend to have more similar names than in average, whereas users with shortest path distances above three tend to have less similar names than in average.

Stay tuned for more results (eg. considering the ReTweet graph and ReTweet frequency) and happy number crunching!

You find challenge-related publications on our new literature page. Contribution is open via BibSonomy. You will have to join the 20DC13 group.