Primed Toolkit¶
NLP functions designed for fast execution times. General utility functions added as required. Currently under development, so use at your own risk, although it is likely to be stable if using pip
.
Installation¶
Simply use pip
:
pip install primed --upgrade
Then, import the module:
import primed as ptk
NLP Examples¶
Text should ideally be cleaned first (i.e. free of punctuation). You can use ptk.clean()
.
Clean text¶
Removes extra spaces, punctuation, and optionally lowers the text. Careful using this if parsing for names, countries, smileys etc.
ptk.clean('Ha, this is fun! YUP!!!', lower=True)
Case-insensitive replace¶
Possibly the fastest method for a case-insensitive replace, tested against both using an arrayed string and using re
.
ptk.ireplace('I want a hIPpo for my birthday', 'hippo', 'giraffe')
Get all (x to y) grams¶
Returns a dictionary of the ngrams with counts. Possibly the fastest method when compared with itertools
, textblob
, sklearn
, and nltk
.
ptk.ngrams('I love cats meow like really really love cats', min_grams=1, max_grams=10)
Create a comma-separated string using the Oxford comma¶
Because this is the grammatically correct way…
ptk.oxfordize(['cats', 'kittens', 'quantum', 'simulation'])
Capitalize i’s¶
Pretty elementary implementation, included just in case.
ptk.capi('i am british, and i also codify things.')
Correct a / an select¶
Uses the Carnegie Mellon University Pronouncing Dictionary (CMUdict), based on the DoD ARPAbet. Currently uses a naive fall-back; a better alternative would be to guess / learn using existing words in CMUdict.
ptk.a('university')
Convert to Wikipedia URI¶
Naive implementation for now, hoping redirects will help with the majority of capitalization issues for words subsequent to the first.
ptk.wiki_uri('DELTA-V Budget')
For a given text, match all elements in a given list¶
Highly-optimized (super-fast) method to return the first, or all occurrences of all elements within a list, in a given text. You also have the option of specifying whether the match must be exact, i.e. equals. Wildcards ?
(exactly one word) and *
(0 or more words, up to max_star
) are implemented. See tests for more examples. Part of our Cythonized codebase at Primed.
ptk.match_elements('Hello, I am having a rather wonderful day today, and I enjoy coding.', ['will not match', 'next one will', 'having * wonderful day', 'rather * wonderful day'], only_first=False, exact_match=False, max_star=3)
Utility Examples¶
Colourful printing¶
ptk.cprint('Text or object to be stringified', style='OK', bold=True, underline=True, newline=True)
Styles available:
'OK': '\033[92m'
'INFO': '\033[94m'
'WARN': '\033[93m'
'ERROR': '\033[91m'
'FATAL': '\033[31m'
Notes¶
keeper
¶
Using string.translate
is quicker than using regular expressions (see https://stackoverflow.com/a/26517161/2178980).