luckydonald-utils

A collection of utilities I use across different python projects.

Documentation

Install

$ pip install luckydonald-utils Or, to install a specific version: $ pip install luckydonald-utils==0.84

Update

Via pip: $ pip install --upgrade luckydonald-utils

Using the package: $ python -m luckydonaldUtils.selfupdate

From python interpreter:

from luckydonaldUtils import selfupdate

Install from source

• Get the source
  • Download a release from PyPi or Github,
  • or clone the latest code from Github: $ git clone https://github.com/luckydonald/luckydonald-utils.git && cd luckydonald-utils
• Navigate into the luckydonaldUtils folder and run $ python setup.py install

Update from source

• If you manually downloaded your version, just follow the install steps again using a new version.
• If you cloned the code, navigate into the luckydonaldUtils folder and run $ git pull && python setup.py install

luckydonaldUtils Documentation:

(not a complete list)

clazzes

Related to class manipulations.

• Singleton: Metaclass to use, if only the one (the same) instance of a class is needed. (Added in v0.35)
• Kwags: Extend and set __FIELDS__ = tuple("a", "b", ...) to be able to use **obj. (v0.72+)

djangos

Utilities for django.

• csrf (v0.22-27. Merged into djangos.requests in v0.28)

• headers (added in v0.23)

  • @header: decorator to add/set a header. (added in v0.24) Use like @header('X-Important-Number', '4458')
  • @headers: decorator to set headers. Use with an dict @headers({'X-Powered-By': 'Magical Unicorns', 'X-Foo': 'Bar!'}) or as kwargs@headers(X_Powered_By="Magical Unicorns", X_Foo="Bar!")
  • @easteregg_headers: Adds my favorite collection of easter egg headers.

• responses (added in v0.25)

  • json_response(status=None, statusText=None, exception=None, content=None): Easier json Responses, also given Exception is rendered as json too.
  • @catch_exception: (v0.26 only) With v0.27, use @render_all_exceptions, @render_DoOutputException or @render_specific_exception(exception_class, exception_render_func=None).
  • @render_all_exceptions: (v0.27+) Catches all exceptions and renders the exception e as HttpResponse(str(e), status=500).
  • @render_DoOutputException: (v0.27+) Like @render_all_exceptions, but only renders a DoOutputException. Useful if something deep in the call stack want to fail with a message.
  • @render_specific_exception(exception_class, exception_render_func=None): (v0.27+) Like @render_DoOutputException, but you can specify the Exception you are expecting. Also you can optionally set a function to render that exception instead of using the default

    def render(response, e):
        return HttpResponse(str(e), status=500)

• requests (v0.28+)

  • check_csrf(request): Manually checks the csrf. Returns True or False. (moved here in v0.28)
  • GET_to_bool(request, key): Parses a GET parameter in the request as bool. "true" becomes True, "false" becomes False, "null" becomes None.

• middelware a collection.

  • access:
    • AllowFromIPMiddleware: v0.31+ Allow only given IPs to access, else raises a Http404 error. Is will be ignored when DEBUG is True, or settings.ALLOW_FROM is None. Set in the settings.py file: ALLOW_FROM = ["134.169.0.0/16"] Include in your MIDDLEWARE_CLASSES: "luckydonaldUtils.djangos.middleware.access.AllowFromIPMiddleware"
  • header:
    • EastereggHeadersMiddleware: v0.34 Sets some funny headers. Include in your MIDDLEWARE_CLASSES: "luckydonaldUtils.djangos.middleware.headers.EastereggHeadersMiddleware"

      exceptions

      Exception related stuff.

• assert_type_or_raise(value, expected_type_clazz_or_tuple, *more_clazzes, exception_clazz=TypeError, parameter_name=None): Since ² v0.53; Added parameter_name parameter in v54. A better assert(isinstance(a, B) because it supports None (as well as some other types except tuple or list), and an nice exception text.

files

Collection of things which are file related. (This structure exists since v0.47)

• files.basics v0.47+
  • mkdir_p(path) like mkdir -p v0.43
  • open_folder(folder_path) tries to open a folder in your system's browser
  • open_file_folder(file_path) tries to open a folder, and select the given file in your system's browser
• files.mime v0.47+
  • guess_extension(mime) Shortcut for getting extension to a given mime string.
  • get_file_mime(file_path=None, file_url=None) Shortcut to get the mime from either
  • get_byte_mime(bytes) Shortcut to get a mime from bytes in a variable.
  • get_file_suffix(file_path=None, file_url=None) This calls get_file_mime() to get the mime, and then calls guess_extension().
• files.name v0.47+
  • do_a_filename(input_file_name) Bad attempt to make file names better, by replacing some characters. This is no escaping.
• files.temp v0.47+
  • gettempdir(temp_folder_name="luckydonald-utils") Gets/creates a folder in the temporary files of the system.
• files.tree v0.47+
  • tree(directory, padding="", print_files=False, level=-1, print_it=True) (New in v0.47)

flasks

Utilities for flask. v0.59+

• flasks.routing
  • route_for: Basically flask.url_for in reverse. You give a url, it gives you the route and the required arguments.
  • get_safe_next: Checks if the specified url in like a ?next=<url> parameter is part of our registered routes, and thus seems safe to redirect to.
• flasks.launcher
  • Launcher: Allow flask's auto-reload to survive syntax errors and similar exceptions, also show them in the browser.

imports

Utilities for importing and import related stuff. v0.77+

• imports.relative
  • relimport: Basically like os.path.relpath, but for dot delimited import paths.

jinja2

Utilities for the jinja templating engine. v0.64+

• filters: Filter for templates.
  • br: Add <br> to linebreaks.

functions

Information about calling functions (Added in v0.35)

• @caller: Functions decorated with this will be called with an call kwarg, containing information about the function itself, and the caller. If the caller could not be fetched correctly, the callers attributes all will be None.

• @cached: v0.49 Decorator to cache function return values (based on *args and **kwargs). You can specify a datetime.timedelta in max_age after which that function will be called again.

• @deprecated: Decorator to mark functions as deprecated. A warning will be logged when the function is used. With v0.47 you can specify a message.

  @deprecated("Reason goes here")
  def foo():
      pass

• @gone: Decorator to mark functions as gone. A NotImplementedError will be emitted when the function is used.

holder

(Added in v0.45)

Caches a result, and returns it. Useful in if statements.

In python it is not possible to store the result of an expression in a variable while being inside of an if:

if (temp=do_something()) == 42:
    foo(temp)

And storing it before is not an option? (you have a very resources-expensive call, or changing values, or are in an elif)

temp = do_something()
temp2 = do_something_else()
if temp == 42:
    foo(temp)
elif temp2:
    foo2(temp2)

Somebody need to hold that result for you:

from luckydonaldUtils.holder import Holder
h = Holder()
if h(do_something()) == 42:
    foo(h())
elif h(do_something_else()):
    foo2(h())

That's what Holder is for.

iterators

All stuff related to list and iterators.

• iter_with_i (iterator, start_i=0): (Added in v0.34) Deprecated since v0.73. Yields a tuple of the iterator result and an integer incrementing each time.

  for iterator_result, i in iter_with_i(["a","b"])

• chunks(iterable, size): (v0.73+) Yields chunks of an iterable, using the slice protocol. That means, the slices are resolved lazy when needed and thus generated, and not loaded up front. End is determined by the actual length of a chunk being less then the specified size.
• chunks_known_length(iterable, size, length=None): (v0.73+) Similar to chunks(...) but uses a for loop with a previously determined max length to do the slicing, instead of the while True if len < size: break loop.

interactions

Interact with the user.

• safe_eval (user_input, no_builtins_object=NoBuiltins(eval_safe_builtin_list, eval_safe_builtin_mapping)): (Added in v0.37) Tries to make a safe execution of user inputted python code. Per default uses a interactions.NoBuiltins object with interactions.eval_safe_builtin_list as allowed commands.

  • user_input: the string
  • no_builtins_object: a NoBuiltins object, initialized with interactions.eval_safe_builtin_list

• NoBuiltins (allowed_builtins, allowed_functions=None, allowed_vars=None): (Added in v0.37) Used to allow custom variables.

  • allowed_builtins: List of allowed buildins (strings)
  • allowed_functions: Dict with names of functions and the functions to be called.
  • allowed_vars: Given variables. A Mapping with a dict.

• eval_safe_builtin_list: (Added in v0.37) A default list of builtin commands/variables considered 'safe'.

• eval_safe_builtin_mapping: (Added in v0.37) A default list of builtin functions/variables, which are not at __builtin__s root level, but mapped as if. They are mathematical functions considered 'safe'.

network

• ip: Tools for ip addresses
  • binary_ip_to_str(host): converts the binary ip to a string.
• mod_wsgi
  • reloader: Auto reloading capabilities for mod_wsgi environments. (v0.28+)

regex

Package of some regular expressions I found useful.

• github
  • REPO_NAME_REGEX: Repo name validation. (Since¹ v0.40)
  • AT_USERNAME_REGEX: Searches @usernames. (Since v0.40; added match group user in v0.41)
  • FILE_URL_REGEX: Matches github urls pointing to files or directories.(Since¹ v0.40; added match group protocol in v54)
  • SIMPLE_URL_REGEX: Matches github urls. (Added in v0.54+) Matching groups:
    • url: the complete url
      • protocol: 'https://' or 'http://' or empty/non-existent
      • user: git user or organisation
      • repo: the repository
      • path: When existent, this is not the project page (root of master)
        • kind: blob or tree
        • branch: the name of the branch (kind=tree), or the commit hash (kind=blob)
      • file: the rest of the filepath (from root of that branch, can be empty)
    • hash: Can be non-existent or empty. Everything behind the '#'
• url
  • URL_REGEX: Matching URLs. Based on dperini's MIT licensed Gist (Added in v0.58)
  • youtube:
    • YOUTUBE_REGEX: Matches youtube videos. The matching group vid contains the video id.

tg_bots

Utilities for the telegram bots (pytgbot and/or teleflask).

• gitinfo: Parse git meta information written by a deploy script.
• language: Helper for loading language files (classes). v0.64+
• peer: Tools for handling chats and users. v0.74+
  • chat: Tools for chats.
    • format: Formatting related chat tools.
      • format_chat(chat: Chat): Formats a channel for html, escaping username and title.
  • user: Tools for chats.
    • format: Formatting related chat tools.
      • format_user(user: User, ...): Formats a user for html, escaping html tags where needed.
      • retrieve_and_format_user(user_id, ...): Retrieves a user from telegram, and formats it with format_user(...).
    • rights: User permission related tools.
      • is_admin(...): Checks if a user has admin privileges, and optionally a specified right.
      • retrieve_and_is_admin(...): Retrieves info about a user in a chat from telegram, and calls is_admin(...).

text

String manipulation, etc.

• split_in_parts(string, parts, strict=False): v0.48+ Splits a string in given parts pieces.

typing

Additions to the great stuff in the typing module. Python 3+ it seems.

• JSONType: The stuff returned by json.loads(str). v0.73+

compat v0.58

Detecting versions and stuff. For string compatibility use .encoding.

• py2: True if is python 2, False otherwise.
• py3: True if is python 3, False otherwise.

webserver

(added in v0.20)

• class BetterHTTPRequestHandler
  • Like the BaseHTTPRequestHandler, but without output to stderr, (why would anyone do that?!?). Instead it goes to loggers. Also some helpfull things.
  • Changed to use logging
    • log_message now uses logger.info
    • log_request writes to logger.debug, to not spam you on every request.
    • log_error uses logger.error
  • Added a write_text(self, msg, content_type="text/plain", is_binary=False) function to make answering with text/data easy.
    • msg: The text to send to the browser/client.
    • content_type: Optional. If you don't like it to be text, change that here.
    • is_binary: Optional. Text (unicode) needs to be converted to binary. If you already have binary (e.g. an PNG as binary data) you can set that here.
    • returns nothing.
  • Modified translate_path: Now accepts local fitting paths automatically, e.g. "/path/to/www-dir/foo.png" is valid if that folder exists. Now it won't change the path to "/path/to/www-dir/foo.png/path/to/www-dir/foo.png", like it did before.
  • Added a parse_POST function to get the post request's data fields returned.
• start_a_webserver(handler, port, host=""). Starts a HTTPServer, using the given handler.
  • handler: An handler instance, e.g. an BetterHTTPRequestHandler
  • port: The port where to serve on. For example 80 or 8080 for HTTP (80 often needs root privileges).
  • host: Optional. A host where to serve on. If an empty string "" (default) is given, all incoming connections are allowed. (you can connect from localhost, from lan, from internet, etc.)
  • returns: The HTTPServer created.

Notes

1. regex.github.REPO_NAME_REGEX and regex.github.FILE_URL_REGEX where already present in v0.38 as regex.urls.github.REPO_NAME_REGEX and regex.urls.github.GITHUB_FILE_REGEX.
2. exceptions.assert_type_or_raise was already present in v0.46 as exceptions.assert_or_raise.
3. **