tokio.connectors.hpss module

Connect to various outputs made available by HPSS

class tokio.connectors.hpss.HpssDailyReport(*args, **kwargs)

Bases: tokio.connectors.common.SubprocessOutputDict

Representation for the daily report that HPSS can generate

load_str(input_str)

Parse the HPSS daily report text

tokio.connectors.hpss._find_columns(line, sep='=', gap=' ', strict=False)

Determine the column start/end positions for a header line separator

Takes a line separator such as the one denoted below:

Host Users IO_GB =============== ===== ========= heart 53 148740.6

and returns a tuple of (start index, end index) values that can be used to slice table rows into column entries.

Parameters:

• line (str) – Text comprised of separator characters and spaces that define the extents of columns
• sep (str) – The character used to draw the column lines
• gap (str) – The character separating sep characters
• strict (bool) – If true, restrict column extents to only include sep characters and not the spaces that follow them.

Returns:

Return type:

list of tuples

tokio.connectors.hpss._hpss_timedelta_to_secs(timedelta_str)

Convert HPSS-encoded timedelta string into seconds

Parameters:timedelta_str (str) – String in form d-HH:MM:SS where d is the number of days, HH is hours, MM minutes, and SS seconds Returns:number of seconds represented by timedelta_str Return type:int

tokio.connectors.hpss._parse_section(lines, start_line=0)

Parse a single table of the HPSS daily report

Converts a table from the HPSS daily report into a dictionary. For example an example table may appear as:

Archive : IO Totals by HPSS Client Gateway (UI) Host
Host             Users      IO_GB       Ops
===============  =====  =========  ========
heart               53   148740.6     27991
dtn11                5    29538.6      1694
Total               58   178279.2     29685
HPSS ACCOUNTING:         224962.6

which will return a dict of form:

{
    "system": "archive",
    "title": "io totals by hpss client gateway (ui) host",
    "records": {
        "heart": {
            "io_gb": "148740.6",
            "ops": "27991",
            "users": "53",
        },
        "dtn11": {
            "io_gb": "29538.6",
            "ops": "1694",
            "users": "5",
        },
        "total": {
            "io_gb": "178279.2",
            "ops": "29685",
            "users": "58",
        }
    ]
}

This function is robust to invalid data, and any lines that do not appear to be a valid table will be treated as the end of the table.

Parameters:

• lines (list of str) – Text of the HPSS report
• start_line (int) –

  Index of lines defined such that

  • lines[start_line] is the table title
  • lines[start_line + 1] is the table heading row
  • lines[start_line + 2] is the line separating the table heading and the first row of data
  • lines[start_line + 3:] are the rows of the table

Returns:

Tuple of (dict, int) where

• dict contains the parsed contents of the table
• int is the index of the last line of the table + 1

Return type:

tuple

tokio.connectors.hpss._rekey_table(table, key)

Converts a list of records into a dict of records

Converts a table of records as returned by _parse_section() of the form:

{
    "records": [
        {
            "host": "heart",
            "io_gb": "148740.6",
            "ops": "27991",
            "users": "53",
        },
        ...
    ]
}

Into a table of key-value pairs the form:

{
    "records": {
        "heart": {
            "io_gb": "148740.6",
            "ops": "27991",
            "users": "53",
        },
        ...
    }
}

Does not handle degenerate keys when re-keying, so only some tables with a uniquely identifying key can be rekeyed.

Parameters:

• table (dict) – Output of the _parse_section() function
• key (str) – Key to pull out of each element of table[‘records’] to use as the key for each record

Returns:

Table with records expressed as key-value pairs instead of a list

Return type:

dict