• functools —- 高阶函数和可调用对象上的操作
    • partial Objects

    functools —- 高阶函数和可调用对象上的操作

    源代码:Lib/functools.py

    functools 模块应用于高阶函数，即——参数或（和）返回值为其他函数的函数。通常来说，此模块的功能适用于所有可调用对象。

    functools 模块定义了以下函数:

    • @functools.cachedproperty(_func)
    • Transform a method of a class into a property whose value is computed onceand then cached as a normal attribute for the life of the instance. Similarto property(), with the addition of caching. Useful for expensivecomputed properties of instances that are otherwise effectively immutable.

    示例:

    1. class DataSet:
    2.     def __init__(self, sequence_of_numbers):
    3.         self._data = sequence_of_numbers
    4.  
    5.     @cached_property
    6.     def stdev(self):
    7.         return statistics.stdev(self._data)
    8.  
    9.     @cached_property
    10.     def variance(self):
    11.         return statistics.variance(self._data)

    3.8 新版功能.

    注解

    This decorator requires that the dict attribute on each instancebe a mutable mapping. This means it will not work with some types, such asmetaclasses (since the dict attributes on type instances areread-only proxies for the class namespace), and those that specifyslots without including dict as one of the defined slots(as such classes don't provide a dict attribute at all).

    • functools.cmpto_key(_func)
    • 将(旧式的)比较函数转换为新式的 key function . 在类似于 sorted() ， min() ， max() ， heapq.nlargest() ， heapq.nsmallest() ， itertools.groupby() 等函数的 key 参数中使用。此函数主要用作将 Python 2 程序转换至新版的转换工具，以保持对比较函数的兼容。

    比较函数意为一个可调用对象，该对象接受两个参数并比较它们，结果为小于则返回一个负数，相等则返回零，大于则返回一个正数。key function则是一个接受一个参数，并返回另一个用以排序的值的可调用对象。

    示例:

    1. sorted(iterable, key=cmp_to_key(locale.strcoll))  # locale-aware sort order

    有关排序示例和简要排序教程，请参阅 排序指南 。

    3.2 新版功能.

    • @functools.lrucache(_user_function)
    • @functools.lrucache(_maxsize=128, typed=False)
    • 一个为函数提供缓存功能的装饰器，缓存 maxsize 组传入参数，在下次以相同参数调用时直接返回上一次的结果。用以节约高开销或I/O函数的调用时间。

    由于使用了字典存储缓存，所以该函数的固定参数和关键字参数必须是可哈希的。

    不同模式的参数可能被视为不同从而产生多个缓存项，例如, f(a=1, b=2) 和 f(b=2, a=1) 因其参数顺序不同，可能会被缓存两次。

    If user_function is specified, it must be a callable. This allows thelru_cache decorator to be applied directly to a user function, leavingthe maxsize at its default value of 128:

    1. @lru_cachedef count_vowels(sentence):    sentence = sentence.casefold()    return sum(sentence.count(vowel) for vowel in 'aeiou')

    如果 maxsize 设置为 None ，LRU功能将被禁用且缓存数量无上限。 maxsize 设置为2的幂时可获得最佳性能。

    如果 typed 设置为true，不同类型的函数参数将被分别缓存。例如， f(3) 和 f(3.0) 将被视为不同而分别缓存。

    为了衡量缓存的有效性以便调整 maxsize 形参，被装饰的函数带有一个 cacheinfo() 函数。当调用 cache_info() 函数时，返回一个具名元组，包含命中次数 _hits，未命中次数 misses ，最大缓存数量 maxsize 和 当前缓存大小 currsize。在多线程环境中，命中数与未命中数是不完全准确的。

    该装饰器也提供了一个用于清理/使缓存失效的函数 cache_clear() 。

    原始的未经装饰的函数可以通过 wrapped 属性访问。它可以用于检查、绕过缓存，或使用不同的缓存再次装饰原始函数。

    “最久未使用算法”（LRU）缓存 在“最近的调用是即将到来的调用的最佳预测因子”时性能最好（比如，新闻服务器上最受欢迎的文章倾向于每天更改）。 “缓存大小限制”参数保证缓存不会在长时间运行的进程比如说网站服务器上无限制的增加自身的大小。

    一般来说，LRU缓存只在当你想要重用之前计算的结果时使用。因此，用它缓存具有副作用的函数、需要在每次调用时创建不同、易变的对象的函数或者诸如time（）或random（）之类的不纯函数是没有意义的。

    静态 Web 内容的 LRU 缓存示例:

    1. @lru_cache(maxsize=32)def get_pep(num):    'Retrieve text of a Python Enhancement Proposal'    resource = 'http://www.python.org/dev/peps/pep-%04d/' % num    try:        with urllib.request.urlopen(resource) as s:            return s.read()    except urllib.error.HTTPError:        return 'Not Found'
    2. >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:…     pep = get_pep(n)…     print(n, len(pep))
    3. >>> get_pep.cache_info()CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)

    以下是使用缓存通过 动态规划 计算 斐波那契数列 的例子。

    1. @lru_cache(maxsize=None)def fib(n):    if n < 2:        return n    return fib(n-1) + fib(n-2)
    2. >>> [fib(n) for n in range(16)][0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]
    3. >>> fib.cache_info()CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)

    3.2 新版功能.

    在 3.3 版更改: 添加 typed 选项。

    在 3.8 版更改: Added the user_function option.

    • @functools.total_ordering
    • 给定一个声明一个或多个全比较排序方法的类，这个类装饰器实现剩余的方法。这减轻了指定所有可能的全比较操作的工作。

    此类必须包含以下方法之一：lt() 、le()、gt() 或 ge()。另外，此类必须支持 eq() 方法。

    例如:

    1. @totalorderingclass Student:    def isvalidoperand(self, other):        return (hasattr(other, "lastname") and                hasattr(other, "firstname"))    def __eq(self, other):        if not self._is_valid_operand(other):            return NotImplemented        return ((self.lastname.lower(), self.firstname.lower()) ==                (other.lastname.lower(), other.firstname.lower()))    def __lt(self, other):        if not self._is_valid_operand(other):            return NotImplemented        return ((self.lastname.lower(), self.firstname.lower()) <                (other.lastname.lower(), other.firstname.lower()))

    注解

    While this decorator makes it easy to create well behaved totallyordered types, it does come at the cost of slower execution andmore complex stack traces for the derived comparison methods. Ifperformance benchmarking indicates this is a bottleneck for a givenapplication, implementing all six rich comparison methods instead islikely to provide an easy speed boost.

    3.2 新版功能.

    在 3.4 版更改: Returning NotImplemented from the underlying comparison function forunrecognised types is now supported.

    • functools.partial(func, /, *args, **keywords)
    • Return a new partial object which when calledwill behave like func called with the positional arguments args_and keyword arguments _keywords. If more arguments are supplied to thecall, they are appended to args. If additional keyword arguments aresupplied, they extend and override keywords.Roughly equivalent to:
    1. def partial(func, /, *args, **keywords):
    2.     def newfunc(*fargs, **fkeywords):
    3.         newkeywords = {**keywords, **fkeywords}
    4.         return func(*args, *fargs, **newkeywords)
    5.     newfunc.func = func
    6.     newfunc.args = args
    7.     newfunc.keywords = keywords
    8.     return newfunc

    The partial() is used for partial function application which "freezes"some portion of a function's arguments and/or keywords resulting in a new objectwith a simplified signature. For example, partial() can be used to createa callable that behaves like the int() function where the base argumentdefaults to two:

    1. >>> from functools import partial
    2. >>> basetwo = partial(int, base=2)
    3. >>> basetwo.__doc__ = 'Convert base 2 string to an int.'
    4. >>> basetwo('10010')
    5. 18
    • class functools.partialmethod(func, /, *args, **keywords)
    • Return a new partialmethod descriptor which behaveslike partial except that it is designed to be used as a methoddefinition rather than being directly callable.

    func must be a descriptor or a callable (objects which are both,like normal functions, are handled as descriptors).

    When func is a descriptor (such as a normal Python function,classmethod(), staticmethod(), abstractmethod() oranother instance of partialmethod), calls to get aredelegated to the underlying descriptor, and an appropriatepartial object returned as the result.

    When func is a non-descriptor callable, an appropriate bound method iscreated dynamically. This behaves like a normal Python function whenused as a method: the self argument will be inserted as the firstpositional argument, even before the args and keywords supplied tothe partialmethod constructor.

    示例:

    1. >>> class Cell(object):
    2. ...     def __init__(self):
    3. ...         self._alive = False
    4. ...     @property
    5. ...     def alive(self):
    6. ...         return self._alive
    7. ...     def set_state(self, state):
    8. ...         self._alive = bool(state)
    9. ...     set_alive = partialmethod(set_state, True)
    10. ...     set_dead = partialmethod(set_state, False)
    11. ...
    12. >>> c = Cell()
    13. >>> c.alive
    14. False
    15. >>> c.set_alive()
    16. >>> c.alive
    17. True

    3.4 新版功能.

    • functools.reduce(function, iterable[, initializer])
    • Apply function of two arguments cumulatively to the items of iterable, fromleft to right, so as to reduce the iterable to a single value. For example,reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5).The left argument, x, is the accumulated value and the right argument, y, isthe update value from the iterable. If the optional initializer is present,it is placed before the items of the iterable in the calculation, and serves asa default when the iterable is empty. If initializer is not given anditerable contains only one item, the first item is returned.

    大致相当于：

    1. def reduce(function, iterable, initializer=None):
    2.     it = iter(iterable)
    3.     if initializer is None:
    4.         value = next(it)
    5.     else:
    6.         value = initializer
    7.     for element in it:
    8.         value = function(value, element)
    9.     return value

    See itertools.accumulate() for an iterator that yields all intermediatevalues.

    • @functools.singledispatch
    • Transform a function into a single-dispatchgeneric function.

    To define a generic function, decorate it with the @singledispatchdecorator. Note that the dispatch happens on the type of the first argument,create your function accordingly:

    1. >>> from functools import singledispatch
    2. >>> @singledispatch
    3. ... def fun(arg, verbose=False):
    4. ...     if verbose:
    5. ...         print("Let me just say,", end=" ")
    6. ...     print(arg)

    To add overloaded implementations to the function, use the register()attribute of the generic function. It is a decorator. For functionsannotated with types, the decorator will infer the type of the firstargument automatically:

    1. >>> @fun.register
    2. ... def _(arg: int, verbose=False):
    3. ...     if verbose:
    4. ...         print("Strength in numbers, eh?", end=" ")
    5. ...     print(arg)
    6. ...
    7. >>> @fun.register
    8. ... def _(arg: list, verbose=False):
    9. ...     if verbose:
    10. ...         print("Enumerate this:")
    11. ...     for i, elem in enumerate(arg):
    12. ...         print(i, elem)

    For code which doesn't use type annotations, the appropriate typeargument can be passed explicitly to the decorator itself:

    1. >>> @fun.register(complex)
    2. ... def _(arg, verbose=False):
    3. ...     if verbose:
    4. ...         print("Better than complicated.", end=" ")
    5. ...     print(arg.real, arg.imag)
    6. ...

    To enable registering lambdas and pre-existing functions, theregister() attribute can be used in a functional form:

    1. >>> def nothing(arg, verbose=False):
    2. ...     print("Nothing.")
    3. ...
    4. >>> fun.register(type(None), nothing)

    The register() attribute returns the undecorated function whichenables decorator stacking, pickling, as well as creating unit tests foreach variant independently:

    1. >>> @fun.register(float)
    2. ... @fun.register(Decimal)
    3. ... def fun_num(arg, verbose=False):
    4. ...     if verbose:
    5. ...         print("Half of your number:", end=" ")
    6. ...     print(arg / 2)
    7. ...
    8. >>> fun_num is fun
    9. False

    When called, the generic function dispatches on the type of the firstargument:

    1. >>> fun("Hello, world.")
    2. Hello, world.
    3. >>> fun("test.", verbose=True)
    4. Let me just say, test.
    5. >>> fun(42, verbose=True)
    6. Strength in numbers, eh? 42
    7. >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
    8. Enumerate this:
    9. 0 spam
    10. 1 spam
    11. 2 eggs
    12. 3 spam
    13. >>> fun(None)
    14. Nothing.
    15. >>> fun(1.23)
    16. 0.615

    Where there is no registered implementation for a specific type, itsmethod resolution order is used to find a more generic implementation.The original function decorated with @singledispatch is registeredfor the base object type, which means it is used if no betterimplementation is found.

    To check which implementation will the generic function choose fora given type, use the dispatch() attribute:

    1. >>> fun.dispatch(float)
    2. <function fun_num at 0x1035a2840>
    3. >>> fun.dispatch(dict)    # note: default implementation
    4. <function fun at 0x103fe0000>

    To access all registered implementations, use the read-only registryattribute:

    1. >>> fun.registry.keys()
    2. dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
    3.           <class 'decimal.Decimal'>, <class 'list'>,
    4.           <class 'float'>])
    5. >>> fun.registry[float]
    6. <function fun_num at 0x1035a2840>
    7. >>> fun.registry[object]
    8. <function fun at 0x103fe0000>

    3.4 新版功能.

    在 3.7 版更改: The register() attribute supports using type annotations.

    • class functools.singledispatchmethod(func)
    • Transform a method into a single-dispatchgeneric function.

    To define a generic method, decorate it with the @singledispatchmethoddecorator. Note that the dispatch happens on the type of the first non-selfor non-cls argument, create your function accordingly:

    1. class Negator:
    2.     @singledispatchmethod
    3.     def neg(self, arg):
    4.         raise NotImplementedError("Cannot negate a")
    5.  
    6.     @neg.register
    7.     def _(self, arg: int):
    8.         return -arg
    9.  
    10.     @neg.register
    11.     def _(self, arg: bool):
    12.         return not arg

    @singledispatchmethod supports nesting with other decorators such as@classmethod. Note that to allow for dispatcher.register,singledispatchmethod must be the outer most decorator. Here is theNegator class with the neg methods being class bound:

    1. class Negator:
    2.     @singledispatchmethod
    3.     @classmethod
    4.     def neg(cls, arg):
    5.         raise NotImplementedError("Cannot negate a")
    6.  
    7.     @neg.register
    8.     @classmethod
    9.     def _(cls, arg: int):
    10.         return -arg
    11.  
    12.     @neg.register
    13.     @classmethod
    14.     def _(cls, arg: bool):
    15.         return not arg

    The same pattern can be used for other similar decorators: staticmethod,abstractmethod, and others.

    3.8 新版功能.

    • functools.updatewrapper(_wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
    • Update a wrapper function to look like the wrapped function. The optionalarguments are tuples to specify which attributes of the original function areassigned directly to the matching attributes on the wrapper function and whichattributes of the wrapper function are updated with the corresponding attributesfrom the original function. The default values for these arguments are themodule level constants WRAPPERASSIGNMENTS (which assigns to the wrapperfunction's module, name, qualname, annotationsand doc, the documentation string) and WRAPPERUPDATES (whichupdates the wrapper function's __dict, i.e. the instance dictionary).

    To allow access to the original function for introspection and other purposes(e.g. bypassing a caching decorator such as lru_cache()), this functionautomatically adds a wrapped attribute to the wrapper that refers tothe function being wrapped.

    The main intended use for this function is in decorator functions whichwrap the decorated function and return the wrapper. If the wrapper function isnot updated, the metadata of the returned function will reflect the wrapperdefinition rather than the original function definition, which is typically lessthan helpful.

    update_wrapper() may be used with callables other than functions. Anyattributes named in assigned or updated that are missing from the objectbeing wrapped are ignored (i.e. this function will not attempt to set themon the wrapper function). AttributeError is still raised if thewrapper function itself is missing any attributes named in updated.

    3.2 新版功能: Automatic addition of the wrapped attribute.

    3.2 新版功能: Copying of the annotations attribute by default.

    在 3.2 版更改: Missing attributes no longer trigger an AttributeError.

    在 3.4 版更改: The wrapped attribute now always refers to the wrappedfunction, even if that function defined a wrapped attribute.(see bpo-17482)

    • @functools.wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
    • This is a convenience function for invoking update_wrapper() as afunction decorator when defining a wrapper function. It is equivalent topartial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated).For example:
    1. >>> from functools import wraps
    2. >>> def my_decorator(f):
    3. ...     @wraps(f)
    4. ...     def wrapper(*args, **kwds):
    5. ...         print('Calling decorated function')
    6. ...         return f(*args, **kwds)
    7. ...     return wrapper
    8. ...
    9. >>> @my_decorator
    10. ... def example():
    11. ...     """Docstring"""
    12. ...     print('Called example function')
    13. ...
    14. >>> example()
    15. Calling decorated function
    16. Called example function
    17. >>> example.__name__
    18. 'example'
    19. >>> example.__doc__
    20. 'Docstring'

    Without the use of this decorator factory, the name of the example functionwould have been 'wrapper', and the docstring of the original example()would have been lost.

    partial Objects

    partial objects are callable objects created by partial(). Theyhave three read-only attributes:

    • partial.func

    • A callable object or function. Calls to the partial object will beforwarded to func with new arguments and keywords.

    • partial.args

    • The leftmost positional arguments that will be prepended to the positionalarguments provided to a partial object call.

    • partial.keywords

    • The keyword arguments that will be supplied when the partial object iscalled.

    partial objects are like function objects in that they arecallable, weak referencable, and can have attributes. There are some importantdifferences. For instance, the name and doc attributesare not created automatically. Also, partial objects defined inclasses behave like static methods and do not transform into bound methodsduring instance attribute look-up.