Section: FSCLI (1)
Updated: December 2018
- fsobjimport [-d directory] [-D delimiter] [-f manifest-file] [-g group] -m mediaid [-M mode] [-p prefix] [-t files|list|media] [-T time] [-u user] [-U] [-W]
- fsobjimport -C [-m mediaid]
- fsobjimport -R -m mediaid
- fsobjimport -s [-F type] [-l] [-m mediaid]
- fsobjimport -S -i requestid|-m mediaid
- fsobjimport -h
fsobjimport(1)is used to import object storage media and to ingest content into the Tertiary Manager system. It also provides administrative operations for scanning and listing the contents of object storage media and for reporting the status of import requests.
The media must be known to the Tertiary Manager system before the import can be requested and must be in the protect state in order for it to be used for import.
For file imports, the contents of the files found on the media will be copied into a destination directory. If the file already exists in the destination directory, a numeric suffix will be appended to the name of the copied file, increasing from zero, until a file with a unique name can be created. Otherwise, option -W can be specified to request that the file be overwritten if it already exists. Once the import has completed, no references to the media are retained and the media can be removed from the Tertiary Manager system unless needed for other purposes.
For media imports, media is retained in the Tertiary Manager system, and the metadata required to map the contents of an object storage bucket into a StorNext file system will be created for each file found on the media. The created metadata allows the data to be retrieved through the Tertiary Manager system. Once the import is complete, the media state must be changed from the protect state to allow access to the files.
Specifying the list import type will generate a list of the files on an object storage medium. The file list will be created in JSON format. The contents of the file can be modified and then used for file or media import.
fsobjimport(1) will log file error information for the current operation to /usr/adic/TSM/logs/reports/import.mediaid. The status information for an import operation can be displayed by specifying the -s option of the fsobjimport(1) command.
If an import request terminates before successful completion, it can be resumed by specifying the -R option with the fsobjimport(1) command. In this case, the -m option must also be specified to identify the failed import request. The import request will be resumed from the failure point.
If an import request has been terminated and is no longer needed, the -U option can be used to undo the import. If neither an undo nor resume of a terminated import is required, the checkpoint metadata for this media should be cleared with the -C option of the fsobjimport(1) command. The -m option must also be specified to identify the import request to clear.
An import request can be tuned by modifying system parameters which define the number of threads used on the request. System parameters IMP_META_NUM_THREADS and IMP_DBASE_NUM_THREADS can be modified to optimize the media import processing for your system. IMP_META_NUM_THREADS defines the number of threads used to create the file metadata and IMP_DBASE_NUM_THREADS the number of threads used to create the database metadata. Parameter IMP_COPY_NUM_THREADS defines the number of threads used to copy data on file imports and IMP_REMOVE_NUM_THREADS defines the number of threads used to remove the imported files or metadata for an undo request. By default, five threads are used for each thread type. For further descriptions of each system parameter, see the fs_sysparm.README file.
- Clear the import request information for the specified medium or for all terminated import requests. After using this option, the import request can no longer be resumed.
- -d directory
- Ingest data into the destination directory directory. The destination directory must be associated with a relation point when ingesting media. The destination directory may be associated with a relation point if ingesting files. Note that if file ingest is chosen and the destination directory is not under a relation point, the files will be ingested but will not be managed by the Tertiary Manager system.
- -D delimiter
- Specifies the character representing a directory delimiter in the object storage names.
- -f manifest-file
- If specified with -t list, output a list of the files on an object storage medium to this file. The file list will be output in JSON format and will include file attributes. If specified with -t media or files, ingest the files listed in file manifest-file. The file attribute values can be modified from their defaults before using the file list for ingest.
- -F type
- Sets the output format to the specified type. Valid values are TEXT (default), XML, or JSON.
- TEXT is the “legacy” textual format.
- XML (Extensible Markup Language) is a set of rules for encoding documents in machine-readable form. XML’s design goals emphasize simplicity, generality, and usability over the Internet. It is a textual data format. Although the design of XML focuses on documents, it is widely used for the representation of arbitrary data structures, for example in web services. You may validate the XML output using the XSD (see fsxsd(1) for details). See http://en.wikipedia.org/wiki/XML for more information.
- -g group
- Set ingested files to group group where group is the group name or numeric group ID. Defaults to 0.
- Print a usage message and exit.
- Specify the request ID of the import request to suspend. The fsobjimport(1) command, specified with options -s and -l, can be used to display the request IDs of pending imports.
- Long status report. For the specified import request, this option provides all of the import option values, the state of the import, import identifiers, and the progress of the import.
- -m mediaid
- Operate on the medium mediaid. The medium must be known to the Tertiary Manager system.
- -M mode
- Set the permission bits for ingested files to mode. If this option is not specified, for file imports, the mode will default to 0440, and for media imports, the mode will be inherited from the destination directory, with the execute bits set to zero.
- -p prefix
- Import only those files beginning with the specified prefix.
- Restart the media or file import processing from the failure point using the checkpoint metadata. The request is identified by the media ID specified with the -m option. Restarts of failed list requests are not supported.
- Display the status of the import request identified by the media ID specified with the -m option, or, in the case where the -m option is omitted, display all pending or failed import requests.
- Suspend an import request. Either the -m option or the -i option must be used to identify the import request to suspend. A suspended import can be restarted by specifying the -R option with the media ID. If an import is no longer needed, the -U option can be specified to undo the partially completed import. If neither an import resume nor undo is needed, the checkpoint metadata should be cleared from the Storage Manager by specifying the -C option with the media ID of the suspended request.
- -t type
- Identify the import request type. Supported types are:
- Copy contents of medium to destination directory
- Generate a list of files residing on medium
- Create metadata for each file residing on medium, which will allow file content to be retrieved
- If the type is files or media, then the -d option must also be specified. If the type is list, an output file must be specified with option -f.
- -T time
- Import only those files created after the specified time. Times are expressed in local time and must be specified in format YYYY-MM-DD HH:MM:SS.
- -u user
- Set ingested files to owner user where user is the user name or numeric user ID. Defaults to 0.
- Undo a media or file import request. An undo of a media import will delete the metadata and database information associated with the files found on the imported media. An undo of a file import will remove all files copied into the destination directory during the import. The options specified with an undo must match those used on the import to return to the pre-import state. If this is an undo of a terminated import, the checkpoint metadata will be used to undo the request.
- On file imports, overwrite a file if it already exists.
Exit codes for the fsobjimport(1) command are:
- Import file or manifest file already exists.
- Bad address specified in file system request.
- The import request was interrupted or suspended.
- Invalid command arguments, manifest file, or object storage data.
- An I/O error occurred while transferring object storage data or when accessing the manifest file or database.
- Object ID or path length exceeds the system maximum.
- Destination path is not a directory.
- The specified media has not been configured, a path to the media has not been configured, the specified import request is not pending, or the destination directory does not exist.
- Insufficient memory to process command.
- No space on file system.
- Object storage import not supported by provider.
- Cannot connect to object storage system or database.
- The media is write-protected, an import request is already active for the media, or you do not have permission to access the object storage system or to access a file.
Import files from medium AWS0000, copying the files to directory /stornext/snfs1/import1. Create the files with an owner ID of 101, a group ID of 120, and with file permissions of 0660. Pre-existing files in the destination directory which have the same name as an imported file will be overwritten:
fsobjimport -m AWS0000 -t files -d /stornext/snfs1/import1 -u 101 \ -g 120 -M 0660 -W
Undo the import request from Example 1, removing all imported files from directory /stornext/snfs1/import1:
fsobjimport -m AWS0000 -t files -d /stornext/snfs1/import1 -u 101 \ -g 120 -M 0660 -W -U
Import medium STS0001 into the Tertiary Manager system, creating the metadata which allows access to the file contents. Import only those files which were created after 7:30 P.M., August 10, 2017:
fsobjimport -m STS0001 -t media -d /stornext/snfs1/import2 \ -T "2017-08-10 19:30:00"
Generate a list of files found on medium AWS0000, which are prefixed with accounts, but do not ingest the contents. Replace the object storage delimiter ‘:’ with a forward slash (‘/’) in the destination file path:
fsobjimport -m AWS0000 -t list -f manifest-file -p accounts -D :
The file created with the list option can then be modified and used to create a manifest file that is used for ingesting content:
fsobjimport -m AWS0000 -t files -d /stornext/snfs1/import3 \ -f manifest-file
Check the status of all running fsobjimport(1) processes:
Suspend the import of medium AWS0000:
fsobjimport -m AWS0000 -S
Restart a suspended import of medium AWS0000:
fsobjimport -m AWS0000 -R
Undo a suspended import of medium AWS0000:
fsobjimport -m AWS0000 -U