SNHISTORY

Section: Maintenance Commands (8)
Updated: October 2018


NAME

snhistory – Display StorNext File System history

SYNOPSIS

snhistory [-F FsName] [-d pathname] [options]

DESCRIPTION

The snhistory program provides the capability to query the metadata archive for the history of file system activity that has transpired between two given points of time. It requires that file system configuration parameter metadataArchive is set to “true”, that metadataArchiveDays is set to a non-zero value, and that metadataArchiveSearch be set to “true”. snhistory can return the history for the entire file system, for any directory and all of its files and sub-directories, or for a single file. The output format is standard JSON by default but can be changed to compact JSON, plain text, or compact text. In the compact text format, the file name is listed without any event association.

The snhistory program can be run in one of two modes: event-history mode or changed-file mode. By default, snhistory runs in event-history mode. To run in changed-file mode, the user must use either the -l or -L option.

In event-history mode, snhistory generates a list of file system events for all files and directories in ascending chronological order. The type of events generated are CREATE, REMOVE, RENAME, MODIFY, LINK, LINKCOUNT, CHMOD, CHOWN, and CHGRP.

In changed-file mode, snhistory generates a list of pathnames for all files in the file system or specified subdirectory that have been either created, deleted, modified, or renamed. The generated list is not in any specific order. For all output formats except compact text, each file listed will be displayed with the most “significant” event for the given time period along with the current size of the file. The most significant events are creates, removes, and renames. So, if a file was modified several times and then deleted during the given time period, the output from snhistory will show that the file was removed.

OPTIONS

-F FsName
This is the name of the StorNext file system. Using this option will cause snhistory to retrieve the activity history from the file system’s active metadata archive.
-D debug_level
This option is for internal use only.
-C
Specifies that the output format should be compact JSON or compact text if the -T option is used. The compact text format only prints the file path names and these path names are relative to the directory specified by the -p option, if present.
-T
Specifies that the output format should be plain text instead of JSON.
-L
Run snhistory in changed-file mode. snhistory will generate a pathname for all files that have been created, removed, renamed, or modified within the specified start and end times.
-l
Same as the -L option except that pathnames will be as of the end time as specified by the -e option or the end TID as specified by the -t option. For example, if a file foo was created in directory foodir and then moved to directory bardir then using this option will cause the corresponding file CREATE event to show a pathname of bardir/foo. In essence, this says that during the specified time period file bardir/foo was created.
-d pathname
This instructs snhistory to query for activity history directly from the metadata archive that exists in the directory specified by pathname rather than sending a request to an activated FSM to query the metadata archive. This option overrides the -F option, so the user should use either -F or -d but not both.
-h
This option causes snhistory to display the command-line usage message.
-H
This option causes snhistory to display a more detailed description of the command-line usage message.
-f filename
This instructs snhistory to return the activity history for the file with the specified name that exists in the directory given by the -p option. If the -p option is not used, then snhistory looks for the given file in the file system root directory.
-m event_mask
This option allows the user to specify the file system activity of interest and is valid only when snhistory is executed in event-history mode. This option takes a character string as an argument. The character string is made from one or more of the following characters:
a – Attribute changes (i.e. linkcount, chmod, chgrp, chown)
c – Creates, renames, hard links
r – Removes
m – Modifications
f – Files
d – DirectoriesFor example, to request the history for all creates, removes, and modifications for files only, use “-m crmf”. To request remove activity for files and directories, use “-m r”.
-n max_results
This option is used to limit the number of events returned from snhistory when executed in event-history mode or the number of files returned when executed in changed-file mode. The argument max_results is specified in units of one thousand. That is, “-n 20” limits the output to 20,000 events (in event-history mode) or files (in changed-file mode) in the output. By default, snhistory returns 50,000 events or files requested within the specified time period. This option can be used to limit the output to a manageable size. If snhistory hits the specified limit before processing all events, it will return a “next” transaction ID (TID) and an “end” TID in the snhistory_summary record that can be used to continue the history query where the previous call left off. Setting max_results to large values can cause snhistory to fail when the FSM exceeds its response time limit or to consume excessive amounts of memory when the -d option is used.
-o output
This specifies the name of the output file. If this option is not used, snhistory writes its output to stdout.
-q filename
Write qustat counters to the named file on completion.
-p pathname
snhistory returns events for all files and directories starting at the StorNext root directory by default. Use this option to limit the query to a particular directory and all of its sub-directories. For example, “-p /foodir” limits the query to directory foodir whose parent is the StorNext file system root directory (i.e. /stornext/snfs1/foodir). The leading “/” character is required. It can also be specified using the full StorNext path, “-p /stornext/snfs1/foodir”. If the full StorNext path is specified then the -F option is unnecessary.
-s date:time
Use this option to specify the starting date and time for the history query. The date:time argument has the format yyyy-mm-dd:hh:mm:ss. For example, “-s 2016-05-12:10:00:00”.
-e date:time
Use this option to specify the ending date and time for the history query. The date:time argument has the format yyyy-mm-dd:hh:mm:ss. For example, “-e 2016-05-12:10:59:59”.
-t sTID[:eTID]
Use this option to specify the starting and ending transaction ID for the history query. This option cannot be used if either the -s or -e options were used to specify the start and end times. This option also has a special case where if only the colon character is specified (i.e. snhistory -F snfs1 -t:), then the last transaction ID that currently exists in the metadata archive will be returned:
root => snhistory -F snfs1 -t:
# LAST_TID: 34568
root =>
-w waittime
Use this option to specify the number of minutes that snhistory should wait for a response from the FSM before timing out. This value can also be specified in snfs_rest_config.json for all instances of this command as part of the process entry named snhistory. This file is installed by default and has a value of 5 minutes.

JSON OUTPUT FORMAT

To demonstrate the format of the JSON output, four directories and one file were created in a new file system. Running snhistory in changed-file mode produces the following output:
root => snhistory -F snfs1 -L
{
“snhistory_summary”: {
“command”: “snhistory -F snfs1 -L”,
“time”: 1464041388177080,
“num_events”: 1
“next_tid”: 136
},
“event_list”: [
{
“type”: “CREATE”,
“time”: 1464032746853778,
“pathname”: “/a/b/d/e/newfile”,
“size”: 29,
“uid”: 0,
“gid”: 0
}
]
}
root =>

The first record is always a snhistory_summary record. At a minimum it contains the command-line that produced the given JSON output, the time when the command was executed, the total number of events returned, and the tid to use as the starting tid for the subsequent snhistory command to retrieve the next set of consecutive events. In this example, one file was created named “newfile”.

The following is an example of using the -n option to limit the number of events returned to the caller to ten thousand:
root => snhistory -F snfs1 -s 2016-05-12:10:00:00 \
-e 2016-05-12:10:59:59 -n 10
{
“snhistory_summary”: {
“command”: “snhistory -F snfs1 -s 2016-05-12:10:00:00 -e
2016-05-12:10:59:59 -n 10”,
“time”: 1464041388177080,
“num_events”: 10006
“next_tid”: 7723321
“end_tid”: 9332588
},
“event_list”: [
. . .
]
}
root =>

A little more than ten thousand events were returned because snhistory queries data at the level of file system transactions and many events can take place within a single transaction. Since the event limit was hit, the snhistory_summary record contains the fields for next_tid and end_tid. To continue with the query, the next call to snhistory would look like:
root => snhistory -F snfs1 -t 7723321:9332588 -n 10

The field end_tid will NOT be included in the snhistory_summary record when the value for next_tid has exceeded end_tid, signifying that all events within the given time period have been returned to the user.

EXAMPLE snhistory COMMANDS

To direct snhistory to search for events or changed files in a metadata archive that is associated with an active file system, the file system must be identified using either the -F or -p options. For example, the following two commands are equivalent:
root => snhistory -F snfs1 -p /foodir/bardir
root => snhistory -p /stornext/snfs1/foodir/bardir

To direct snhistory to search a metadata archive that is not associated with an active file system, use the -d option rather than the -F or -p options:
root => snhistory -d /tmp/snfs1-mdarchive -p /foodir/bardir

To query for the metadata history for file app001.dat in directory /foodir/bardir of file system snfs2:
root => snhistory -p /stornext/snfs2/foodir/bardir -f app001.dat

Here is an example of using snhistory in event-history mode. Query for all events within the given two hour time period using the text output format:
root => snhistory -F rhSource -T -s 2017-02-13:14:30:00 \
-e 2017-02-13:16:30:00
[02-15-2017 12:36:55][snhistory -F rhSource -T     -s 2017-02-13:14:30:00 -e 2017-02-13:16:30:00]     num_events:9 next_tid:193
[02-13-2017 14:42:30][159] REMOVE /dir1/file12
[02-13-2017 14:42:30][159] REMOVE /dir1
[02-13-2017 14:42:30][159] REMOVE /dir1/file11
[02-13-2017 15:35:26][162] CREATE /foo
[02-13-2017 15:35:28][164] MODIFY /foo 4
[02-13-2017 15:44:44][170] CREATE /bar
[02-13-2017 15:44:46][172] MODIFY /bar 4
[02-13-2017 15:46:12][179] CREATE /soothe.txt
[02-13-2017 15:46:14][181] MODIFY /soothe.txt 418
root =>

The same example but using snhistory in changed-file mode. Query for all files that have changed during the same 2 hour time period above using the compact-text output format:
root => snhistory -F rhSource -LCT -s 2017-02-13:14:30:00 \
-e 2017-02-13:16:30:00
/dir1/file12
/dir1/file11
/foo
/bar
/soothe.txt
# NEXT_TID: 193
root =>

To query for all files created or renamed in the file system within the given one hour period using the compact JSON output format:
root => snhistory -F snfs2 -C -m fc -s 2017-02-13:14:00:00 \
-e 2017-02-13:14:59:59

EXIT STATUS

snhistory will return zero on success and non-zero on failure.

FILES

/usr/cvfs/data/FsName/FsName-mdarchive
/usr/cvfs/config/FsName.cfgx

SEE ALSO

snfs_rest_config.json(4)