Thursday, April 28, 2011

Best way to turn word list into frequency dict

What's the best way to convert a list/tuple into a dict where the keys are the distinct values of the list and the values are the the frequencies of those distinct values?

In other words:

['a', 'b', 'b', 'a', 'b', 'c']
--> 
{'a': 2, 'b': 3, 'c': 1}

(I've had to do something like the above so many times, is there anything in the standard lib that does it for you?)

EDIT:

Jacob Gabrielson points out there is something coming in the standard lib for the 2.7/3.1 branch

From stackoverflow
  • Kind of

    from collections import defaultdict
    fq= defaultdict( int )
    for w in words:
        fq[w] += 1
    

    That usually works nicely.

  • I have to share an interesting but kind of ridiculous way of doing it that I just came up with:

    >>> class myfreq(dict):
    ...     def __init__(self, arr):
    ...         for k in arr:
    ...             self[k] = 1
    ...     def __setitem__(self, k, v):
    ...         dict.__setitem__(self, k, self.get(k, 0) + v)
    ... 
    >>> myfreq(['a', 'b', 'b', 'a', 'b', 'c'])
    {'a': 2, 'c': 1, 'b': 3}
    
    John Fouhy : (self.get(k) or 0) can be better written as self.get(k, 0)
  • This is an abomination, but:

    from itertools import groupby
    dict((k, len(list(xs))) for k, xs in groupby(sorted(items)))
    

    I can't think of a reason one would choose this method over S.Lott's, but if someone's going to point it out, it might as well be me. :)

    ʞɔıu : points for cleverness
  • I find that the easiest to understand (while might not be the most efficient) way is to do:

    {i:words.count(i) for i in set(words)}
    
    S.Lott : +1: Got to get me some of that Python 3.0 syntactic sugar.
    Trey Stout : That is pretty hot
    Nathan Ross Powell : Beautiful Python!
  • Just a note that, starting with Python 2.7/3.1, this functionality will be built in to the collections module, see this bug for more information. Here's the example from the release notes:

    >>> from collections import Counter
    >>> c=Counter()
    >>> for letter in 'here is a sample of english text':
    ...   c[letter] += 1
    ...
    >>> c
    Counter({' ': 6, 'e': 5, 's': 3, 'a': 2, 'i': 2, 'h': 2,
    'l': 2, 't': 2, 'g': 1, 'f': 1, 'm': 1, 'o': 1, 'n': 1,
    'p': 1, 'r': 1, 'x': 1})
    >>> c['e']
    5
    >>> c['z']
    0
    
    ʞɔıu : looks even simpler than that, looks like you can just pass the string to the Counter constructor and it does it for you

0 comments:

Post a Comment