How do I use iRODS with IDrive® e2?

Integrated Rule-Oriented Data System (iRODS) has been verified for use with IDrive® e2. Configure this policy-based open source data management software with IDrive® e2 to store and manage huge volumes of complex and sensitive data sets from iRODS.



Using iRODS with IDrive® e2 you can:

  1. Store massive volumes of complex and crucial data sets from iRODS.
  2. Manage and utilize the stored iRODS data sets for research, commercial, and governmental organizations across the globe.

Prerequisites:

  1. An active IDrive® e2 account. Sign up here if you do not have one.
  2. A bucket in IDrive® e2. Learn how to create a bucket, add objects to it, and assign public access to it.
  3. IDrive® e2 Access Key and Secret Access Key. Learn how to create an access key.
  4. iRODS Software Version 4.2.11
  5. iRODS S3 Plugin Version 4.2.11.3

Install iRODS S3 Plugin

Install the latest version of the iRODS S3 Plugin on all iRODS servers.

Run the following command:

sudo apt-get install irods-resource-plugin-s3

Save your IDrive® e2 access key and secret key

Save your IDrive® e2 access key and secret access key pair into a file accessible by the iRODS server. The file should contain the access key and secret access key, one on each of two separate lines, respectively.

In the following example, the access key and secret access keys are saved to:

/var/lib/irods/idrivee2.keypair

Create an iRODS S3 resource connecting to IDrive® e2 backend

Create a resource that connects to IDrive® e2 using the following pattern:

iadmin mkresc <resc_name> s3 <server_name>:/<bucket_name> <configuration context>

The example below creates an iRODS resource named IDrivee2resc1. It assumes the target bucket is named mybucket, the connection will use the secure 'https' protocol to connect with f0o2.tx12.idrivee2-3.com in the Dallas region, and that the keypair is stored in /var/lib/irods/idrivee2.keypair:

iadmin mkresc idrivee2resc1 s3 `hostname`:/mybucket
“S3_DEFAULT_HOSTNAME=f0o2.tx12.idrivee2-3.com;S3_AUTH_FILE=/
var/lib/irods/idrivee2.keypair;S3_REGIONNAME=dallas;S3_RETRY
_COUNT=2;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTPS;HOST_MODE=cac
heless_attached;S3_SIGNATURE_VERSION=4;S3_ENABLE_MPU=1;S3_MP
U_THREADS=30”

Note: This example shows the use of IDrive® e2's Dallas storage region. To use other storage regions, use the appropriate IDrive® e2 endpoint URL. Learn how to view the endpoints for the IDrive® e2 regions.

Test a dataset into the iRODS resource connected to IDrive® e2

Test put/get of a file into the iRODS resource IDrivee2resc1.

$ echo 'this is a test' > test.txt
$ iput -R idrivee2resc1 test.txt
$ iget test.txt -

If you were in the rods home collection in iRODS, the file should be visible in IDrive® e2 at /mybucket/home/rods/test.txt.

Note - This example is for test purpose.