quotish

Overview

quotish is a combinator library for building command lines and shell snippets using PEP 750 template strings.

>>> import quotish as qsh
>>> name = 'arcana shadow'
>>> echo = qsh.cmd(t'echo "Hello, {name}!"')
>>> echo.render()
"echo 'Hello, arcana shadow!'"
>>> echo.argv()
['echo', 'Hello, arcana shadow!']
>>> qsh.cmd(t'sh -c {echo}').argv()
['sh', '-c', "echo 'Hello, arcana shadow!'"]

There are three primary functions that take template strings:

• cmd(): Parses the template as shell syntax into individual arguments, which are then re-joined when calling Renderable.render(). Alternatively, you can get a list of arguments with Command.argv(), which can then be passed to subprocess.run or other similar functions.

  This is the function you usually want to use.

  As a bonus, you can concatenate Command instances with + or += to combine their argument lists:

  >>> (qsh.cmd(t'echo') + t'hello' + qsh.cmd('world!')).argv()
  ['echo', 'hello', 'world!']
  

  If you find yourself only calling Command.argv() on the result, without using the Command for anything else, you can use the argv() shorthand:

  >>> qsh.argv(t'echo "Hello, {name}!"')
  ['echo', 'Hello, arcana shadow!']
  

• shell():

  Parses the template as shell syntax but does not split or modify it in any way other than expanding the template variables and added the needed quotes. This is useful if you want to use actual shell syntax, which would normally be re-quoted by cmd() when it re-joins the arguments.

• plain():

  Interpolates the template variables but does not perform any quoting or interpret the template string in any way. This solely exists as a way to get normal f-string behavior while working with Renderables, but you should avoid using this to build actual command lines.

These all return instances of Renderable subclasses, so you can use Renderable.render() on any of them (but of course, Command.argv() is specific to cmd()’s return value).

Lists given to cmd() or shell() can be expanded by using unpack() for simple expansion, aliased as the * format specifier (if you don’t care about type safety, since format specifiers are not type-checked):

>>> qsh.cmd(t'echo +{qsh.unpack([1, 2])}+{[3,4]:*}+').argv()
['echo', '+1', '2+3', '4+']

or product() / ^ for a brace-expansion-style cartesian product:

>>> qsh.cmd(t'echo +{qsh.product([1, 2])}+{[3, 4]:^}+').argv()
['echo', '+1+3+', '+1+4+', '+2+3+', '+2+4+']

Passing a Command to either of these will unpack its arguments:

>>> cmd = qsh.cmd(t'echo a b c')
>>> qsh.shell(t'exec {qsh.unpack(cmd)}').render()
'exec echo a b c'

(These functions are only intended to be used directly in the t-string, i.e. their values should not be saved anywhere, for readability purposes—it’s confusing if what appears to be a single value is interpolated and somehow expands into multiple, without this being clear at the call site.)

To mask out secret values when debugging, use mask() with Renderable.debug():

>>> password = qsh.mask('password123')
>>> cmd = qsh.cmd(t'echo {password}')
>>> str(cmd)
"<<echo '[masked]'>>"
>>> cmd.debug()
"echo '[masked]'"
>>> cmd.render()
'echo password123'

Core types

class quotish.Renderable

The type returned by the various combinators in this module, offering rendering with or without unmasking. (See mask() for information on masking values.)

Additionally, any Renderables can be concatenated with Parts (i.e. another Renderable or a plain str) using the & operator:

>>> import quotish as qsh
>>> (qsh.plain('abc') & qsh.plain('def') & 'ghi').render()
'abcdefghi'

This is as an alias for join().

debug() → str

Render this to a string, without unmasking any mask()‘d values.

render() → str

Render this to a string, unmasking any mask()‘d values in the process.

To keep the values masked, use debug() instead.

type quotish.Part = Renderable | str

A part of a string that can be passed to various combinators.

While Renderable is what’s usually returned from functions like cmd(), Part‘s inclusion of str allows you to pass plain strings to functions like mask().

Template string handlers

Command lines

class quotish.Command

A command line, internally split following shell quoting rules.

Returned by cmd(), with extra rendering methods specifically for conviently dealing with argument lists.

Unlike other Renderables, Commands can be added to each other or to other t-strings, which will concatenate their argument lists:

>>> import quotish as qsh
>>> echo = qsh.cmd('echo')
>>> echo += t'some arguments'
>>> echo += qsh.cmd(t'more arguments')
>>> echo.argv()
['echo', 'some', 'arguments', 'more', 'arguments']

argv(*, debug: bool = False) → list[str]

Renders the individual command line arguments into a list of strings.

Set debug=True to mask secrets given to mask().

>>> import quotish as qsh
>>> secret = qsh.mask('super secret')
>>> echo = qsh.cmd(t'echo "Your password is: {secret}"')
>>> echo.argv()
['echo', 'Your password is: super secret']
>>> echo.argv(debug=True)
['echo', 'Your password is: [masked]']

quotish.cmd(tmpl: Template) → Command

Parses the given template into an argument list using shell quoting rules.

>>> import quotish as qsh
>>> qsh.cmd(t'echo hello "a name"').argv()
['echo', 'hello', 'a name']
>>> qsh.cmd(t'echo hello "a name"').render()
"echo hello 'a name'"

Variables substituted into the template will be treated as if they were quoted:

>>> name = 'arcana shadow'
>>> qsh.cmd(t'echo hello {name}').argv()
['echo', 'hello', 'arcana shadow']
>>> qsh.cmd(t'echo hello {name}').render()
"echo hello 'arcana shadow'"

You can use unpack() or product() to unpack sequences as individual arguments, or their less-type-safe format specifiers * and ^:

>>> names = ['mystique', 'answer']
>>> qsh.cmd(t'echo "some names: " ({qsh.unpack(names)})').argv()
['echo', 'some names: ', '(mystique', 'answer)']
>>> qsh.cmd(t'echo "some names: " ({names:*})').render()
"echo 'some names: ' '(mystique' 'answer)'"
>>> qsh.cmd(t'ls users/{qsh.product(names)}').argv()
['ls', 'users/mystique', 'users/answer']
>>> qsh.cmd(t'ls users/{names:^}').render()
'ls users/mystique users/answer'

Because this splits and re-joins the string, special characters will be quoted when rendered, quotes will be rewritten to a preferred form, and backslashes will be evaluated:

>>> qsh.cmd(rt"special & 'characters' \"").render()
'special \'&\' characters \'"\''

If you want to substitute values into a template without affecting the literal parts, use shell(), which doesn’t perform the splitting.

quotish.argv(tmpl: Template) → list[str]

An alias for calling cmd() and then argv() on the result.

>>> import quotish as qsh
>>> qsh.argv(t'echo 123')
['echo', '123']
>>> qsh.cmd(t'echo 123').argv()
['echo', '123']

Shell strings

quotish.shell(tmpl: Template) → Renderable

Parses the given template into a shell string using shell quoting rules.

>>> import quotish as qsh
>>> qsh.shell(t'echo hello "a name"').render()
'echo hello "a name"'

Variables substituted into the template will be treated as if they were quoted:

>>> name = 'arcana shadow'
>>> qsh.shell(t'echo hello {name}').render()
"echo hello 'arcana shadow'"

You can use unpack() or product() to unpack sequences as individual arguments, or their less-type-safe format specifiers * and ^:

>>> names = ['mystique!', 'answer!']
>>> qsh.shell(rt"echo 'some names: ' \({qsh.unpack(names)}\)").render()
"echo 'some names: ' \\('mystique!' 'answer!'\\)"
>>> qsh.shell(t'ls users/{qsh.product(names)}').render()
"ls users/'mystique!' users/'answer!'"

Unlike cmd(), shell() does not touch any characters outside of the substituted values:

>>> qsh.shell(rt"special & 'characters' \"").render()
'special & \'characters\' \\"'

However, in order to substitute in values, strings may be closed and reopened via intermediate quotes:

>>> name1 = 'mystique'
>>> name2 = 'arcana shadow'
>>> qsh.shell(t'echo "Hello, {name1}!" "and {name2}" too').render()
'echo "Hello, "mystique"!" "and "\'arcana shadow\' too'

Because it ignores the meanings of any characters other than quotes, you may have surprising behavior when mixing special characters with cartesian products:

>>> qsh.shell(t'echo {names:^}; echo 123').render()
"echo 'mystique!'; 'answer!'; echo 123"

In an actual shell, the semicolon is a syntactic element and thus is not part of the arguments to echo, but here it gets treated as a regular character.

If you want an argument list for use with functions like subprocess.run, use cmd() instead.

quotish.unsafe(inner: Part | _Unpack | _Product) → _Unsafe

Renders the inner part without any quoting in shell().

>>> import quotish as qsh
>>> qsh.shell(t'echo {'hello!'}').render()  # quoted
"echo 'hello!'"
>>> qsh.shell(t'echo {qsh.unsafe('hello!')}').render()  # NOT quoted
'echo hello!'

Can also be used to wrap unpack() or product():

>>> dangerous = ['mystique!', 'answer!']
>>> qsh.shell(t'echo {qsh.unsafe(qsh.unpack(dangerous))}').render()
'echo mystique! answer!'
>>> qsh.shell(t'echo {qsh.unsafe(qsh.product(dangerous))}').render()
'echo mystique! answer!'

Plain text

quotish.plain(tmpl: Template) → Renderable

Formats the given template as if a plain string, with no special quoting.

>>> import quotish as qsh
>>> name = 'arcana shadow'
>>> qsh.plain(t'hello, {name}').render()
'hello, arcana shadow'

Unpacking sequences

quotish.unpack(parts: Iterable[object] | Command, *, unsafe_by: str = ' ') → _Unpack

Unpacks the given iterable or command into a cmd() or shell().

>>> import quotish as qsh
>>> names = ['mystique', 'answer']
>>> (echo := qsh.cmd(t'echo {qsh.unpack(names)}')).argv()
['echo', 'mystique', 'answer']
>>> qsh.shell(t'exec {qsh.unpack(echo)}').render()
'exec echo mystique answer'

If unsafe_by is given, it will be used to concatenate the items instead of spaces, but without it being treated as being quoted. Thus, any spaces or other special characters will behave as they would if placed directly in the t-string:

>>> qsh.cmd(t'echo {qsh.unpack(names, unsafe_by=' & ')}').render()
"echo mystique '&' answer"
>>> qsh.cmd(t'echo {names[0]} & {names[1]}').render()  # same as this
"echo mystique '&' answer"
>>> qsh.shell(t'echo {qsh.unpack(names, unsafe_by=' & ')}').render()
'echo mystique & answer'
>>> qsh.shell(t'echo {names[0]} & {names[1]}').render()  # same as this
'echo mystique & answer'

If you want the separator and the items to be quoted, use join().

When used with shell(), if you’d like to ensure neither of them are quoted, use unsafe():

>>> dangerous = ['mystique!', 'answer!']
>>> qsh.shell(t'echo {qsh.unpack(dangerous, unsafe_by=' & ')}').render()
"echo 'mystique!' & 'answer!'"
>>> qsh.shell(t'echo {qsh.unsafe(qsh.unpack(dangerous, unsafe_by=' & '))}').render()
'echo mystique! & answer!'

Unlike regular shell brace expansion, this does not perform a cartesian product; the first and last item will simply be placed next to any other text part of the same shell argument:

>>> qsh.cmd(t'echo ({qsh.unpack(names)})').argv()
['echo', '(mystique', 'answer)']

If you want a cartesian product, use product().

quotish.product(parts: Iterable[object] | Command) → _Product

Unpacks the given iterable or command into a cmd() or shell(), using a cartesian product.

>>> import quotish as qsh
>>> names = ['mystique', 'answer']
>>> (echo := qsh.cmd(t'echo {qsh.product(names)},{qsh.product(names)}')).argv()
['echo', 'mystique,mystique', 'mystique,answer', 'answer,mystique', 'answer,answer']
>>> qsh.shell(t'exec {qsh.unpack(echo)}').render()
'exec echo mystique,mystique mystique,answer answer,mystique answer,answer'

This is similar to shell brace expansion. If instead, you want to simply expand the items side-by-side, use unpack().

Masking

quotish.mask(inner: Part) → _Masked

Masks the given part when rendered with Renderable.debug().

>>> import quotish as qsh
>>> secret = 'my super secret password'
>>> echo = qsh.cmd(t'echo {qsh.mask(secret)}')
>>> echo.debug()
"echo '[masked]'"
>>> echo.render()
"echo 'my super secret password'"
>>> str(echo)  # uses debug(), not render()
"<<echo '[masked]'>>"

Generally, you should try to avoid passing secrets through the command line, but if absolutely necessary, this should help make logging safer.

quotish.unmask(inner: Part) → _Unmasked

Unmasks any masked values in the given part.

>>> import quotish as qsh
>>> m = qsh.mask('my secret')
>>> m.debug()
'[masked]'
>>> qsh.unmask(m).debug()
'my secret'

Miscellaneous combinators

quotish.join(parts: Iterable[Part], *, by: str) → _Join

Joins the given parts with the given separator.

When later expanded into cmd() or shell(), the entire, joined result is treated as a single argument and thus will be escaped as such.

>>> import quotish as qsh
>>> joined = qsh.join(['hello', 'world'], by=' ')
>>> joined.render()
'hello world'
>>> qsh.cmd(t'echo {joined}').render()
"echo 'hello world'"

You can also use & as an alias for joining two Parts by an empty separator:

>>> (qsh.plain('hello') & 'world').render()
'helloworld'

quotish.quote(inner: Part) → _Quoted

Shell quotes the given part.

>>> import quotish as qsh
>>> qsh.quote('hello world!').render()
"'hello world!'"

You should prefer using cmd() or shell() when possible for more exhaustive functionality. (They both use quote() internally.)