Script helpers¶
releng-tool provides a series of helper functions which can be used in script-based packages, post-processing and more. Helper functions provided are listed below:
Available functions¶
- releng_tool.debug(msg, *args)
Logs a debug message to standard out with a trailing new line. By default, debug messages will not be output to standard out unless the instance is configured with debugging enabled.
debug('this is a debug message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.err(msg, *args)
Logs an error message to standard error with a trailing new line and (if enabled) a red colorization.
err('this is an error message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.hint(msg, *args)
Logs a hint message to standard out with a trailing new line and (if enabled) a cyan colorization.
hint('this is a hint message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.log(msg, *args)
Logs a (normal) message to standard out with a trailing new line.
log('this is a message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.note(msg, *args)
Logs a notification message to standard out with a trailing new line and (if enabled) an inverted colorization.
note('this is a note message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.releng_cat(file, *args)
Attempts to read one or more files provided to this call. For each file, it will be read and printed out to the standard output.
An example when using in the context of script helpers is as follows:
releng_cat('my-file')
- Parameters:
file – the file
*args (optional) – additional files to include
- Returns:
True
if all the files exists and were printed to the standard output;False
if one or more files could not be read
- releng_tool.releng_copy(src, dst, quiet=False, critical=True, dst_dir=None)
This call will attempt to copy a provided file or directory, defined by
src
into a destination file or directory defined bydst
. Ifsrc
is a file, thendst
is considered to be a file or directory; ifsrc
is a directory,dst
is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unlessquiet
is set toTrue
). Ifcritical
is set toTrue
and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit
).An example when using in the context of script helpers is as follows:
# (stage) # my-file releng_copy('my-file', 'my-file2') # (stage) # my-file # my-file2 releng_copy('my-file', 'my-directory/') # (stage) # my-directory/my-file # my-file # my-file2 releng_copy('my-directory/', 'my-directory2/') # (stage) # my-directory/my-file # my-directory2/my-file # my-file # my-file2
- Parameters:
src – the source directory or file
dst – the destination directory or file* (*if
src
is a file)quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
dst_dir (optional) – force hint that the destination is a directory
- Returns:
True
if the copy has completed with no error;False
if the copy has failed- Raises:
SystemExit – if the copy operation fails with
critical=True
- releng_tool.releng_copy_into(src, dst, quiet=False, critical=True)
This call will attempt to copy a provided file or directory, defined by
src
into a destination directory defined bydst
. If a target directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unlessquiet
is set toTrue
). Ifcritical
is set toTrue
and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit
).An example when using in the context of script helpers is as follows:
# (stage) # my-file releng_copy_into('my-file', 'my-directory') # (stage) # my-directory/my-file # my-file releng_copy_into('my-directory', 'my-directory2') # (stage) # my-directory/my-file # my-directory2/my-file # my-file
- Parameters:
src – the source directory or file
dst – the destination directory
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
- Returns:
True
if the copy has completed with no error;False
if the copy has failed- Raises:
SystemExit – if the copy operation fails with
critical=True
- releng_tool.releng_env(key, value=<object object>)
Provides a caller a simple method to fetch or configure an environment variable for the current context. This call is the same as if one directly fetched from or managed a key-value with
os.environ
. Ifvalue
is not provided, the environment variable’s value (if set) will be returned. Ifvalue
is set to a value ofNone
, any set environment variable will be removed.An example when using in the context of script helpers is as follows:
# get an environment variable value = releng_env('KEY') # set an environment variable releng_env('KEY', 'VALUE')
- Parameters:
key – the environment key
value (optional) – the environment value to set
- Returns:
the value of the environment variable
- releng_tool.releng_execute(args, cwd=None, env=None, env_update=None, quiet=None, critical=True, poll=False, capture=None, expand=True)
Runs the command described by
args
until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory incwd
. The execution request will returnTrue
on a successful execution;False
if an issue has been detected (e.g. bad options or called process returns a non-zero value). In the event that the execution fails, an error message will be output to standard error unlessquiet
is set toTrue
.The environment variables used on execution can be manipulated in two ways. First, the environment can be explicitly controlled by applying a new environment content using the
env
dictionary. Key of the dictionary will be used as environment variable names, whereas the respective values will be the respective environment variable’s value. Ifenv
is not provided, the existing environment of the executing context will be used. Second, a caller can instead update the existing environment by using theenv_update
option. Likeenv
, the key-value pairs match to respective environment key-value pairs. The difference with this option is that the call will use the original environment values and update select values which match in the updated environment request. Whenenv
andenv_update
are both provided,env_update
will be updated the options based off ofenv
instead of the original environment of the caller.If
critical
is set toTrue
and the execution fails for any reason, this call will issue a system exit (SystemExit
). By default, the critical flag is enabled (i.e.critical=True
).In special cases, an executing process may not provide carriage returns/new lines to simple output processing. This can lead the output of a process to be undesirably buffered. To workaround this issue, the execution call can instead poll for output results by using the
poll
option with a value ofTrue
. By default, polling is disabled with a value ofFalse
.A caller may wish to capture the provided output from a process for examination. If a list is provided in the call argument
capture
, the list will be populated with the output provided from an invoked process.An example when using in the context of script helpers is as follows:
releng_execute(['echo', '$TEST'], env={'TEST': 'this is a test'})
- Parameters:
args – the list of arguments to execute
cwd (optional) – working directory to use
env (optional) – environment variables to use for the process
env_update (optional) – environment variables to append for the process
quiet (optional) – whether or not to suppress output (defaults to
False
)critical (optional) – whether or not to stop execution on failure (defaults to
True
)poll (optional) – force polling stdin/stdout for output data (defaults to
False
)capture (optional) – list to capture output into
expand (optional) – perform variable expansion on arguments
- Returns:
True
if the execution has completed with no error;False
if the execution has failed- Raises:
SystemExit – if the execution operation fails with
critical=True
- releng_tool.releng_execute_rv(command, *args, **kwargs)
Runs the command
command
with providedargs
until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory incwd
. The execution request will return the command’s return code as well as any captured output.The environment variables used on execution can be manipulated in two ways. First, the environment can be explicitly controlled by applying a new environment content using the
env
dictionary. Key of the dictionary will be used as environment variable names, whereas the respective values will be the respective environment variable’s value. Ifenv
is not provided, the existing environment of the executing context will be used. Second, a caller can instead update the existing environment by using theenv_update
option. Likeenv
, the key-value pairs match to respective environment key-value pairs. The difference with this option is that the call will use the original environment values and update select values which match in the updated environment request. Whenenv
andenv_update
are both provided,env_update
will be updated the options based off ofenv
instead of the original environment of the caller.An example when using in the context of script helpers is as follows:
rv, out = releng_execute_rv('echo', '$TEST', env={'TEST': 'env-test'})
- Parameters:
command – the command to invoke
*args (optional) – arguments to add to the command
**cwd – working directory to use
**env – environment variables to use for the process
**env_update – environment variables to append for the process
**expand – perform variable expansion on arguments
- Returns:
the return code and output of the execution request
- releng_tool.releng_exists(path, *args)
Allows a caller to verify the existence of a file on the file system. This call will return
True
if the path exists;False
otherwise.An example when using in the context of script helpers is as follows:
if releng_exists('my-file'): print('the file exists') else: print('the file does not exist')
- Parameters:
path – the path
*args – additional path parts
- Returns:
True
if the path exists;False
otherwise
- releng_tool.releng_exit(msg=None, code=None)
Provides a convenience method to help invoke a system exit call without needing to explicitly use
sys
. A caller can provide a message to indicate the reason for the exit. The provide message will output to standard error. The exit code, if not explicit set, will vary on other arguments. If a message is provided to this call, the default exit code will be1
. If no message is provided, the default exit code will be0
. In any case, if the caller explicitly sets a code value, the provided code value will be used.An example when using in the context of script helpers is as follows:
releng_exit('there was an error performing this task')
- Parameters:
msg (optional) – error message to print
code (optional) – exit code; defaults to 0 if no message or defaults to 1 if a message is set
- Raises:
SystemExit – always raised
- releng_tool.releng_expand(obj, kv=None)
This expand utility method will attempt to expand variables in detected string types. For a detected string which contains substrings in the form of
$value
or${value}
, these substrings will be replaced with their respective key-value (if provided) or environment variable value. For substrings which do not have a matching variable value, the substrings will be replaced with an empty value. If a dictionary is provided, keys and values will be checked if they can be expanded on. If a list/set is provided, each value which be checked if it can be expanded on. If a dictionary key is expanded to match another key, a key-value pair can be dropped. If a set may result in a smaller set if expanded values result in duplicate entries.An example when using in the context of script helpers is as follows:
import os ... os.environ['MY_ENV'] = 'my-environment-variable' value = releng_expand('$MY_ENV') print(value) # will output: my-environment-variable
- Parameters:
obj – the object
kv (optional) – key-values pairs to use
- Returns:
the expanded object
- releng_tool.releng_include(file_path)
The provided call will execute code at the provided file path. The path will be relative to the caller’s script, unless an absolute path is provided. The executed script will be initialized with globals matching the caller’s script.
An example when using in the context of script helpers is as follows:
# load "my-other-script" found alongside the current script releng_include('my-other-script')
- Parameters:
file_path – the script to invoke
- releng_tool.releng_join(path, *paths)
An alias for
os.path.join
. See also https://docs.python.org/library/os.path.html#os.path.join.
- releng_tool.releng_ls(dir_)
Attempts to read a directory for its contents and prints this information to the configured standard output stream.
An example when using in the context of script helpers is as follows:
releng_ls('my-dir/')
- Parameters:
dir – the directory
- Returns:
True
if the directory could be read and its contents have been printed to the standard output;False
if the directory could not be read
- releng_tool.releng_mkdir(dir_, *args, **kwargs)
Attempts to create the provided directory. If the directory already exists, this method has no effect. If the directory does not exist and could not be created, this method will return
None
. Also, if an error has been detected, an error message will be output to standard error (unlessquiet
is set toTrue
).An example when using in the context of script helpers is as follows:
if releng_mkdir('my-directory'): print('directory was created') else: print('directory was not created') target_dir = releng_mkdir(TARGET_DIR, 'sub-folder') if target_dir: # output] target directory: <target>/sub-folder print('target directory:', target_dir) else: print('directory was not created')
- Parameters:
dir – the directory
*args (optional) – additional components of the directory
**quiet (optional) – whether or not to suppress output (defaults to
False
)**critical (optional) – whether or not to stop execution on failure (defaults to
False
)
- Returns:
the directory that exists;
None
if the directory could not be created
- releng_tool.releng_move(src, dst, quiet=False, critical=True, dst_dir=None)
This call will attempt to move a provided file or directory’s contents, defined by
src
into a destination file or directory defined bydst
. Ifsrc
is a file, thendst
is considered to be a file or directory; ifsrc
is a directory,dst
is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created.In the event that a file or directory could not be moved, an error message will be output to standard error (unless
quiet
is set toTrue
). Ifcritical
is set toTrue
and the specified file/directory could not be moved for any reason, this call will issue a system exit (SystemExit
).An example when using in the context of script helpers is as follows:
# (input) # my-directory/another-file # my-file # my-file2 releng_move('my-file', 'my-file3') releng_move('my-directory/', 'my-directory2/') releng_move('my-file2', 'my-directory2/') # (output) # my-directory2/another-file # my-directory2/my-file2 # my-file3
- Parameters:
src – the source directory or file
dst – the destination directory or file* (*if
src
is a file)quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
dst_dir (optional) – force hint that the destination is a directory
- Returns:
True
if the move has completed with no error;False
if the move has failed- Raises:
SystemExit – if the copy operation fails with
critical=True
- releng_tool.releng_move_into(src, dst, quiet=False, critical=True)
This call will attempt to move a provided file or directory’s contents, defined by
src
into a destination directory defined bydst
. If a target directory directory does not exist, it will be automatically created.In the event that a file or directory could not be moved, an error message will be output to standard error (unless
quiet
is set toTrue
). Ifcritical
is set toTrue
and the specified file/directory could not be moved for any reason, this call will issue a system exit (SystemExit
).An example when using in the context of script helpers is as follows:
# (input) # my-directory/another-file # my-file # my-file2 releng_move('my-file', 'my-file3') releng_move('my-directory', 'my-directory2') releng_move('my-file2', 'my-directory2') # (output) # my-directory2/another-file # my-directory2/my-file2 # my-file3
- Parameters:
src – the source directory or file
dst – the destination directory
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
- Returns:
True
if the move has completed with no error;False
if the move has failed- Raises:
SystemExit – if the copy operation fails with
critical=True
- releng_tool.releng_remove(path, quiet=False)
Attempts to remove the provided path if it exists. The path value can either be a directory or a specific file. If the provided path does not exist, this method has no effect. In the event that a file or directory could not be removed due to an error other than unable to be found, an error message will be output to standard error (unless
quiet
is set toTrue
).An example when using in the context of script helpers is as follows:
releng_remove('my-file') # (or) releng_remove('my-directory/')
- Parameters:
path – the path to remove
quiet (optional) – whether or not to suppress output
- Returns:
True
if the path was removed or does not exist;False
if the path could not be removed from the system
- releng_tool.releng_require_version(version, quiet=False, critical=True)
Enables a caller to explicitly check for a required releng-tool version. Invoking this function with a dotted-separated
version
string, the string will be parsed and compared with the running releng-tool version. If the required version is met, this method will have no effect. In the event that the required version is not met, the exceptionSystemExit
will be raised if the critical flag is set; otherwise this call will returnFalse
.An example when using in the context of script helpers is as follows:
# ensure we are using releng-tool v1 releng_require_version('1.0.0')
- Parameters:
version – dotted-separated version string
quiet (optional) – whether or not to suppress output
critical (optional) – whether or not to stop execution on failure
- Returns:
True
if the version check is met;False
if the version check has failed- Raises:
SystemExit – if the version check fails with
critical=True
- releng_tool.releng_tmpdir(dir_=None)
Creates a temporary directory in the provided directory
dir_
(or system default, is not provided). This is a context-supported call and will automatically remove the directory when completed. If the provided directory does not exist, it will created. If the directory could not be created, anFailedToPrepareBaseDirectoryError
exception will be thrown.An example when using in the context of script helpers is as follows:
with releng_tmpdir() as dir_: print(dir_)
- Parameters:
dir (optional) – the directory to create the temporary directory in
- Raises:
FailedToPrepareBaseDirectoryError – the base directory does not exist and could not be created
- releng_tool.releng_touch(file)
Attempts to update the access/modifications times on a file. If the file does not exist, it will be created. This utility call operates in the same fashion as the
touch
system command.An example when using in the context of script helpers is as follows:
if releng_touch('my-file'): print('file was created') else: print('file was not created')
- Parameters:
file – the file
- Returns:
True
if the file was created/updated;False
if the file could not be created/updated
- releng_tool.releng_wd(dir_)
Moves the current context into the provided working directory
dir
. When returned, the original working directory will be restored. If the provided directory does not exist, it will created. If the directory could not be created, anFailedToPrepareWorkingDirectoryError
exception will be thrown.An example when using in the context of script helpers is as follows:
with releng_wd('my-directory/'): # invoked in 'my-directory' # invoked in original working directory
- Parameters:
dir – the target working directory
- Raises:
FailedToPrepareWorkingDirectoryError – the working directory does not exist and could not be created
- releng_tool.success(msg, *args)
Logs a success message to standard error with a trailing new line and (if enabled) a green colorization.
success('this is a success message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.verbose(msg, *args)
Logs a verbose message to standard out with a trailing new line and (if enabled) an inverted colorization. By default, verbose messages will not be output to standard out unless the instance is configured with verbosity.
verbose('this is a verbose message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- releng_tool.warn(msg, *args)
Logs a warning message to standard error with a trailing new line and (if enabled) a purple colorization.
warn('this is a warning message')
- Parameters:
msg – the message
*args – an arbitrary set of positional and keyword arguments used when generating a formatted message
- Raises:
RelengToolWarningAsError – when warnings-are-errors is configured
Importing helpers¶
Scripts directly invoked by releng-tool will automatically have these helpers registered in the script’s globals module (i.e. no import is necessary). If a project defines custom Python modules in their project and wishes to take advantage of these helper functions, the following import can be used to, for example, import a specific function:
from releng_tool import releng_execute
Or, if desired, all helper methods can be imported at once:
from releng_tool import *