Script helpers¶

releng-tool provides a series of helper functions which can be used in script-based packages, post-processing and more. Helper functions provided are listed below:

Available functions¶

releng_tool.debug(msg, *args)¶

Logs a debug message to standard out with a trailing new line. By default, debug messages will not be output to standard out unless the instance is configured with debugging enabled.

debug('this is a debug message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.err(msg, *args)¶

Logs an error message to standard error with a trailing new line and (if enabled) a red colorization.

err('this is an error message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.hint(msg, *args)¶

Added in version 0.13.

Geändert in Version 1.4: Ensure availability in script helpers.

Logs a hint message to standard out with a trailing new line and (if enabled) a cyan colorization.

hint('this is a hint message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.log(msg, *args)¶

Logs a (normal) message to standard out with a trailing new line.

log('this is a message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.note(msg, *args)¶

Logs a notification message to standard out with a trailing new line and (if enabled) an inverted colorization.

note('this is a note message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.releng_cat(file: str | bytes | PathLike, *args: str | bytes | PathLike) → bool¶

Added in version 0.11.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Attempts to read one or more files provided to this call. For each file, it will be read and printed out to the standard output.

An example when using in the context of script helpers is as follows:

releng_cat('my-file')

Parameter:

file – the file
*args (optional) – additional files to include

Rückgabe:

True if all the files exists and were printed to the standard output; False if one or more files could not be read

Geändert in Version 0.12: Add support for dst_dir.

Geändert in Version 1.4: Add support for nested.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

This call will attempt to copy a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination file or directory defined by dst. If src is a file, then dst is considered to be a file or directory; if src is a directory, dst is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (stage)
# my-file
releng_copy('my-file', 'my-file2')
# (stage)
# my-file
# my-file2
releng_copy('my-file', 'my-directory/')
# (stage)
# my-directory/my-file
# my-file
# my-file2
releng_copy('my-directory/', 'my-directory2/')
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-file
# my-file2

Parameter:

src – the source directory or file
dst – the destination directory or file* (*if src is a file)
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
dst_dir (optional) – force hint that the destination is a directory
nested (optional) – nest a source directory into the destination

Rückgabe:

True if the copy has completed with no error; False if the copy has failed

Verursacht:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_copy_into(src: str | bytes | PathLike, dst: str | bytes | PathLike, *, quiet: bool = False, critical: bool = True, nested: bool = False) → bool¶

Added in version 0.13.

Geändert in Version 1.4: Add support for nested.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

This call will attempt to copy a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination directory defined by dst. If a target directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (stage)
# my-file
releng_copy_into('my-file', 'my-directory')
# (stage)
# my-directory/my-file
# my-file
releng_copy_into('my-directory', 'my-directory2')
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-file
releng_copy_into('my-directory', 'my-directory3', nested=True)
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-directory3/directory/my-file
# my-file

Parameter:

src – the source directory or file
dst – the destination directory
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
nested (optional) – nest a source directory into the destination

Rückgabe:

True if the copy has completed with no error; False if the copy has failed

Verursacht:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_env(key, value=<object object>)¶

Added in version 0.3.

Provides a caller a simple method to fetch or configure an environment variable for the current context. This call is the same as if one directly fetched from or managed a key-value with os.environ. If value is not provided, the environment variable’s value (if set) will be returned. If value is set to a value of None, any set environment variable will be removed.

An example when using in the context of script helpers is as follows:

# get an environment variable
value = releng_env('KEY')

# set an environment variable
releng_env('KEY', 'VALUE')

Parameter:

key – the environment key
value (optional) – the environment value to set

Rückgabe:

the value of the environment variable

releng_tool.releng_execute(args, cwd=None, env=None, env_update=None, quiet=None, critical=True, poll=False, capture=None, expand=True, args_str=False, ignore_stderr=False)¶

Geändert in Version 1.13: Add support for expand.

Geändert in Version 1.14: Add support for args_str.

Geändert in Version 2.0: Add support for ignore_stderr.

Runs the command described by args until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory in cwd. The execution request will return True on a successful execution; False if an issue has been detected (e.g. bad options or called process returns a non-zero value). In the event that the execution fails, an error message will be output to standard error unless quiet is set to True.

The environment variables used on execution can be manipulated in two ways. First, the environment can be explicitly controlled by applying a new environment content using the env dictionary. Key of the dictionary will be used as environment variable names, whereas the respective values will be the respective environment variable’s value. If env is not provided, the existing environment of the executing context will be used. Second, a caller can instead update the existing environment by using the env_update option. Like env, the key-value pairs match to respective environment key-value pairs. The difference with this option is that the call will use the original environment values and update select values which match in the updated environment request. When env and env_update are both provided, env_update will be updated the options based off of env instead of the original environment of the caller.

If critical is set to True and the execution fails for any reason, this call will issue a system exit (SystemExit). By default, the critical flag is enabled (i.e. critical=True).

In special cases, an executing process may not provide carriage returns/new lines to simple output processing. This can lead the output of a process to be undesirably buffered. To workaround this issue, the execution call can instead poll for output results by using the poll option with a value of True. By default, polling is disabled with a value of False.

A caller may wish to capture the provided output from a process for examination. If a list is provided in the call argument capture, the list will be populated with the output provided from an invoked process.

An example when using in the context of script helpers is as follows:

releng_execute(['echo', '$TEST'], env={'TEST': 'this is a test'})

Parameter:

args – the list of arguments to execute
cwd (optional) – working directory to use
env (optional) – environment variables to use for the process
env_update (optional) – environment variables to append for the process
quiet (optional) – whether or not to suppress output (defaults to False)
critical (optional) – whether or not to stop execution on failure (defaults to True)
poll (optional) – force polling stdin/stdout for output data (defaults to False)
capture (optional) – list to capture output into
expand (optional) – perform variable expansion on arguments
args_str (optional) – invoke arguments as a single string
ignore_stderr (optional) – ignore any stderr output

Rückgabe:

True if the execution has completed with no error; False if the execution has failed

Verursacht:

SystemExit – if the execution operation fails with critical=True

releng_tool.releng_execute_rv(command, *args, **kwargs)¶

Added in version 0.8.

Geändert in Version 1.13: Add support for expand.

Geändert in Version 1.14: Add support for args_str.

Geändert in Version 2.0: Add support for ignore_stderr.

Runs the command command with provided args until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory in cwd. The execution request will return the command’s return code as well as any captured output.

An example when using in the context of script helpers is as follows:

rv, out = releng_execute_rv('echo', '$TEST', env={'TEST': 'env-test'})

Parameter:

command – the command to invoke
*args (optional) – arguments to add to the command
**cwd – working directory to use
**env – environment variables to use for the process
**env_update – environment variables to append for the process
**expand – perform variable expansion on arguments
**args_str – invoke arguments as a single string
**ignore_stderr – ignore any stderr output

Rückgabe:

the return code and output of the execution request

releng_tool.releng_exists(path: str | bytes | PathLike, *args: str | bytes | PathLike) → bool¶

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Allows a caller to verify the existence of a file on the file system. This call will return True if the path exists; False otherwise.

An example when using in the context of script helpers is as follows:

if releng_exists('my-file'):
    print('the file exists')
else:
    print('the file does not exist')

Parameter:

path – the path
*args (optional) – additional components of the file

Rückgabe:

True if the path exists; False otherwise

releng_tool.releng_exit(msg=None, code=None)¶

Provides a convenience method to help invoke a system exit call without needing to explicitly use sys. A caller can provide a message to indicate the reason for the exit. The provide message will output to standard error. The exit code, if not explicit set, will vary on other arguments. If a message is provided to this call, the default exit code will be 1. If no message is provided, the default exit code will be 0. In any case, if the caller explicitly sets a code value, the provided code value will be used.

An example when using in the context of script helpers is as follows:

releng_exit('there was an error performing this task')

Parameter:

msg (optional) – error message to print
code (optional) – exit code; defaults to 0 if no message or defaults to 1 if a message is set

Verursacht:

SystemExit – always raised

releng_tool.releng_expand(obj, kv=None)¶

This expand utility method will attempt to expand variables in detected string types. For a detected string which contains substrings in the form of $value or ${value}, these substrings will be replaced with their respective key-value (if provided) or environment variable value. For substrings which do not have a matching variable value, the substrings will be replaced with an empty value. If a dictionary is provided, keys and values will be checked if they can be expanded on. If a list/set is provided, each value which be checked if it can be expanded on. If a dictionary key is expanded to match another key, a key-value pair can be dropped. If a set may result in a smaller set if expanded values result in duplicate entries.

An example when using in the context of script helpers is as follows:

import os
...

os.environ['MY_ENV'] = 'my-environment-variable'
value = releng_expand('$MY_ENV')
print(value)
# will output: my-environment-variable

Parameter:

obj – the object
kv (optional) – key-values pairs to use

Rückgabe:

the expanded object

releng_tool.releng_include(file_path: str | bytes | PathLike) → None¶

Added in version 0.12.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

The provided call will execute code at the provided file path. The path will be relative to the caller’s script, unless an absolute path is provided. The executed script will be initialized with globals matching the caller’s script.

An example when using in the context of script helpers is as follows:

# load "my-other-script" found alongside the current script
releng_include('my-other-script')

Parameter:: file_path – the script to invoke

releng_tool.releng_join(path: str | bytes | PathLike, *paths: str | bytes | PathLike)¶: An alias for os.path.join. See also https://docs.python.org/library/os.path.html#os.path.join.

releng_tool.releng_ls(dir_: str | bytes | PathLike, *, recursive: bool = False) → bool¶

Added in version 0.11.

Geändert in Version 2.0: Add support for recursive.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Attempts to read a directory for its contents and prints this information to the configured standard output stream.

An example when using in the context of script helpers is as follows:

releng_ls('my-dir/')

Parameter:

dir – the directory
recursive (optional) – recursive search of entries

Rückgabe:

True if the directory could be read and its contents have been printed to the standard output; False if the directory could not be read

Added in version 0.3.

Geändert in Version 0.13: Add support for critical.

Geändert in Version 1.3: Accepts multiple paths components.

Geändert in Version 1.3: Returns the created path.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Attempts to create the provided directory. If the directory already exists, this method has no effect. If the directory does not exist and could not be created, this method will return None. Also, if an error has been detected, an error message will be output to standard error (unless quiet is set to True).

An example when using in the context of script helpers is as follows:

if releng_mkdir('my-directory'):
    print('directory was created')
else:
    print('directory was not created')

target_dir = releng_mkdir(TARGET_DIR, 'sub-folder')
if target_dir:
    # output] target directory: <target>/sub-folder
    print('target directory:', target_dir)
else:
    print('directory was not created')

Parameter:

dir – the directory
*args (optional) – additional components of the directory
**quiet (optional) – whether or not to suppress output (defaults to False)
**critical (optional) – whether or not to stop execution on failure (defaults to False)

Rückgabe:

the directory that exists; None if the directory could not be created

Geändert in Version 0.14: Add support for dst_dir.

Geändert in Version 1.4: Add support for nested.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

This call will attempt to move a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination file or directory defined by dst. If src is a file, then dst is considered to be a file or directory; if src is a directory, dst is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created.

In the event that a file or directory could not be moved, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be moved for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (input)
# my-directory/another-file
# my-file
# my-file2
releng_move('my-file', 'my-file3')
releng_move('my-directory/', 'my-directory2/')
releng_move('my-file2', 'my-directory2/')
# (output)
# my-directory2/another-file
# my-directory2/my-file2
# my-file3

Parameter:

src – the source directory or file
dst – the destination directory or file* (*if src is a file)
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
dst_dir (optional) – force hint that the destination is a directory
nested (optional) – nest a source directory into the destination

Rückgabe:

True if the move has completed with no error; False if the move has failed

Verursacht:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_move_into(src: str | bytes | PathLike, dst: str | bytes | PathLike, *, quiet: bool = False, critical: bool = True, nested: bool = False) → bool¶

Added in version 0.14.

Geändert in Version 1.4: Add support for nested.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

This call will attempt to move a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination directory defined by dst. If a target directory directory does not exist, it will be automatically created.

An example when using in the context of script helpers is as follows:

# (input)
# my-directory/another-file
# my-directory2/another-file2
# my-file
# my-file2
releng_move_into('my-file', 'my-file3')
releng_move_into('my-directory', 'my-directory3')
releng_move_into('my-file2', 'my-directory3')
releng_move_into('my-directory2', 'my-directory3', nested=True)
# (output)
# my-directory3/my-directory2/another-file2
# my-directory3/another-file
# my-directory3/my-file2
# my-file3

Parameter:

src – the source directory or file
dst – the destination directory
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
nested (optional) – nest a source directory into the destination

Rückgabe:

True if the move has completed with no error; False if the move has failed

Verursacht:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_path(*pathsegments)¶: Added in version 2.3.

An alias for pathlib.Path. See also https://docs.python.org/3/library/pathlib.html#pathlib.Path.

releng_tool.releng_remove(path: str | bytes | PathLike, quiet=False) → bool¶

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Attempts to remove the provided path if it exists. The path value can either be a directory or a specific file. If the provided path does not exist, this method has no effect. In the event that a file or directory could not be removed due to an error other than unable to be found, an error message will be output to standard error (unless quiet is set to True).

An example when using in the context of script helpers is as follows:

releng_remove('my-file')
# (or)
releng_remove('my-directory/')

Parameter:

path – the path to remove
quiet (optional) – whether or not to suppress output

Rückgabe:

True if the path was removed or does not exist; False if the path could not be removed from the system

releng_tool.releng_symlink(target: str | bytes | PathLike, link_path: str | bytes | PathLike, *, quiet: bool = False, critical: bool = True, lpd: bool = False, relative: bool = True) → bool¶

Bemerkung

this call may not work in Windows environments if the user invoking releng-tool does not have permission to create symbolic links.

Added in version 1.4.

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

This call will attempt to create a symbolic link to a target path. A provided link_path determines where the symbolic link will be created. By default, the link_path identifies the symbolic link file to be created. However, if lpd is set to True, it will consider the link_path as a directory to create a symbolic link in. For this case, the resulting symbolic link’s filename will match the basename of the provided target path.

If a symbolic link already exists at the provided link path, the symbolic link will be replaced with the new target. If the resolved link path already exists and is not a symbolic link, this operation will not attempt to create a symbolic link.

If the directory structure of a provided link_path does not exist, it will be automatically created. In the event that a symbolic link or directory could not be created, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the symbolic link could not be created for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# my-link -> my-file
releng_symlink('my-file', 'my-link')

# my-link -> container/my-file
releng_symlink('container/my-file', 'my-link')

# some/folder/my-link -> ../../my-file
releng_symlink('my-file', 'some/folder/my-link')

# my-file -> container/my-file
releng_symlink('my-file', 'container', ldp=True)

# some-path/my-link -> <workdir>/folder/my-file
releng_symlink('folder/my-file', 'some-path/my-link', relative=False)

Parameter:

target – the source path to link to
link_path – the symbolic link path
quiet (optional) – whether to suppress output
critical (optional) – whether to stop execution on failure
lpd (optional) – hint that the link is a directory to create in
relative (optional) – whether to build a relative link

Rückgabe:

True if the symbolic link has been created; False if the symbolic link could not be created

Verursacht:

SystemExit – if the symbolic link operation fails with critical=True

releng_tool.releng_require_version(minver=None, **kwargs)¶

Added in version 0.11.

Geändert in Version 2.0: Support maximum version.

Enables a caller to explicitly check for a required releng-tool version. Invoking this function with a dotted-separated version string, the string will be parsed and compared with the running releng-tool version. If the required version is met, this method will have no effect. In the event that the required version is not met, the exception SystemExit will be raised if the critical flag is set; otherwise this call will return False.

An example when using in the context of script helpers is as follows:

# ensure we are using releng-tool v1
releng_require_version('1.0.0')

Parameter:

minver – dotted-separated minimum version string
**quiet (optional) – whether or not to suppress output
**critical (optional) – whether or not to stop execution on failure
**maxver (optional) – dotted-separated maximum version string

Rückgabe:

True if the version check is met; False if the version check has failed

Verursacht:

SystemExit – if the version check fails with critical=True

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Geändert in Version 2.2: Add support for *args and **wd.

Creates a temporary directory in the provided directory dir_ (or system default, is not provided). This is a context-supported call and will automatically remove the directory when completed. If the provided directory does not exist, it will created. If the directory could not be created, an FailedToPrepareBaseDirectoryError exception will be thrown.

An example when using in the context of script helpers is as follows:

with releng_tmpdir() as dir_:
    print(dir_)

Parameter:

dir (optional) – the directory to create the temporary directory in
**wd (optional) – whether to use the temporary directory as the working directory (defaults to False)

Verursacht:

FailedToPrepareBaseDirectoryError – the base directory does not exist and could not be created

releng_tool.releng_touch(file: str | bytes | PathLike, *args: str | bytes | PathLike) → bool¶

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Geändert in Version 2.2: Add support for *args.

Attempts to update the access/modifications times on a file. If the file does not exist, it will be created. This utility call operates in the same fashion as the touch system command.

An example when using in the context of script helpers is as follows:

if releng_touch('my-file'):
    print('file was created')
else:
    print('file was not created')

Parameter:

file – the file
*args (optional) – additional components of the file

Rückgabe:

True if the file was created/updated; False if the file could not be created/updated

releng_tool.releng_wd(dir_: str | bytes | PathLike) → Iterator[str]¶

Geändert in Version 2.2: Accepts a str, bytes or os.PathLike.

Moves the current context into the provided working directory dir. When returned, the original working directory will be restored. If the provided directory does not exist, it will created. If the directory could not be created, an FailedToPrepareWorkingDirectoryError exception will be thrown.

An example when using in the context of script helpers is as follows:

with releng_wd('my-directory/'):
    # invoked in 'my-directory'

# invoked in original working directory

Parameter:: dir – the target working directory
Verursacht:: FailedToPrepareWorkingDirectoryError – the working directory does not exist and could not be created

releng_tool.success(msg, *args)¶

Logs a success message to standard error with a trailing new line and (if enabled) a green colorization.

success('this is a success message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.verbose(msg, *args)¶

Logs a verbose message to standard out with a trailing new line and (if enabled) an inverted colorization. By default, verbose messages will not be output to standard out unless the instance is configured with verbosity.

verbose('this is a verbose message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.warn(msg, *args)¶

Logs a warning message to standard error with a trailing new line and (if enabled) a purple colorization.

warn('this is a warning message')

Parameter:

msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message

Verursacht:

RelengToolWarningAsError – when warnings-are-errors is configured

Available variables¶

The following variables are registered in the global context for any project or package definition/script.

`releng_args`¶

Geändert in Version 2.5: Variable no longer populated when action-pkg-exec is set.

Bemerkung

When using <pkg>-exec "<cmd>", the releng_args variable will not be populated. This is to prevent conflicts from project-specific argument processing and package-specific run arguments.

A list of arguments forwarded into a releng-tool invoke. If a caller uses the -- argument, all trailing arguments will be populated into „forwarded argument“ list. A project may use these arguments for their own tailoring.

A user can use Python’s argparse module to manage custom arguments. For example, if trying to add a new flag --custom to a build, the following can be added into a project’s releng-tool.rt definition:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--custom', action='store_true')
args = parser.parse_args(args=releng_args)
print(args.custom)

The flag can be enabled by invoking releng-tool using:

releng-tool -- --custom

`releng_version`¶

The version of releng-tool.

print(f'Using version {releng_version}')

Importing helpers¶

Scripts directly invoked by releng-tool will automatically have these helpers registered in the script’s globals module (i.e. no import is necessary). If a project defines custom Python modules in their project and wishes to take advantage of these helper functions, the following import can be used to, for example, import a specific function:

from releng_tool import releng_execute

Or, if desired, all helper methods can be imported at once:

from releng_tool import *

Script helpers¶

Available functions¶

Available variables¶

releng_args¶

releng_version¶

Importing helpers¶

`releng_args`¶

`releng_version`¶