Package com.emc.object.s3.jersey

Class S3JerseyClient

  • All Implemented Interfaces:
    S3Client
    Direct Known Subclasses:
    S3EncryptionClient
    public class S3JerseyClient
extends AbstractJerseyClient
implements S3Client
    Reference implementation of S3Client.

    This implementation uses the JAX-RS reference implementation (Jersey) as it's REST client. When sending or receiving data, the following content handlers are supported by default. Be sure to use the appropriate content-type associated with each object type or the handlers will not understand the request.

    Object Type (class)Expected Content-Type(s)
    byte[]*any*
    java.lang.String*any*
    java.io.File (send-only)*any*
    java.io.InputStream (send-only)*any*
    any annotated JAXB root element beantext/xml, application/xml

    Also keep in mind that you can always send/receive byte[] and do your own conversion.

    To use, simply pass a new S3Config object to the constructor like so: 

         // for client-side load balancing and direct connection to all nodes
     //   single-VDC (client will auto-discover the remaining nodes):
     S3Config config1 = new S3Config(Protocol.HTTP, "10.10.10.11", "10.10.10.12");
     //   multiple VDCs (client will auto-discover remaining nodes in specified VDCs):
     Vdc boston = new Vdc("10.10.10.11", "10.10.10.12").withName("Boston");
     Vdc seattle = new Vdc("10.20.20.11", "10.20.20.12").withName("Seattle");
     S3Config config2 = new S3Config(Protocol.HTTPS, boston, seattle);

     // to use a load balancer will full wildcard DNS setup
     S3Config config3 = new S3Config(new URI("https://s3.company.com")).withUseVHost(true);

     // in all cases, you need to provide your credentials
     configX.withIdentity("my_full_token_id").withSecretKey("my_secret_key");
     S3Client s3Client = new S3JerseyClient(configX);

    To create an object, simply pass the object in to one of the putObject methods. The object type must be one of the supported types above. 

         String stringContent = "Hello World!";
     s3Client.putObject("my-bucket", "my-key", stringContent, "text/plain");

     File fileContent = new File( "spreadsheet.xls" );
     s3Client.putObject("my-bucket", "my-data", fileContent, "application/vnd.ms-excel");

     byte[] binaryContent;
     ... // load binary content to store as an object
     s3Client.putObject("my-bucket", "my-bits", binaryContent, null ); // default content-type is application/octet-stream

    To read an object, specify the type of object you want to receive from a readObject method. The same rules apply to this type. 

         String stringContent = s3Client.readObject("my-bucket", "my-key", String.class);

     byte[] fileContent = s3Client.readObject("my-bucket", "my-data", byte[].class);
     // do something with file content (stream to client? save in local filesystem?)

     byte[] binaryContent = s3Client.readObject("my-bucket", "my-bits", byte[].class);

    Performance

    If you are experiencing performance issues, you might try tuning Jersey's IO buffer size, which defaults to 8k. 

         System.setProperty(ReaderWriter.BUFFER_SIZE_SYSTEM_PROPERTY, "" + 128 * 1024); // 128k
    You can also try using Jersey's URLConnectionClientHandler, but be aware that this handler does not support Expect: 100-Continue behavior if that is important to you. You should also increase http.maxConnections to match your thread count. 
         System.setProperty("http.maxConnections", "" + 32); // if you have 32 threads
     S3Client s3Client = new S3JerseyClient(configX, new URLConnectionClientHandler());

    • Field Detail

      • client

        protected com.sun.jersey.api.client.Client client

      • loadBalancer

        protected com.emc.rest.smart.LoadBalancer loadBalancer

    • Constructor Detail

      • S3JerseyClient

        public S3JerseyClient​(S3Config s3Config)

      • S3JerseyClient

        public S3JerseyClient​(S3Config config,
                      com.sun.jersey.api.client.ClientHandler clientHandler)
        Provide a specific Jersey ClientHandler implementation (default is ApacheHttpClient4Handler). If you experience performance problems, you might try using URLConnectionClientHandler, but note that it will not support the Expect: 100-Continue header and upload size is limited to 2GB. Also note that when using that handler, you should set the "http.maxConnections" system property to match your thread count (default is only 5).

    • Method Detail

      • finalize

        protected void finalize()
                 throws java.lang.Throwable
        Overrides:
        finalize in class java.lang.Object
        Throws:
        java.lang.Throwable

      • shutdown

        public void shutdown()
        Deprecated.
        (2.0.3) use destroy() instead
        Specified by:
        shutdown in interface S3Client

      • destroy

        public void destroy()
        Destroy the client. Any system resources associated with the client will be cleaned up.

        This method must be called when there are not responses pending otherwise undefined behavior will occur.

        The client must not be reused after this method is called otherwise undefined behavior will occur.

        Specified by:
        destroy in interface S3Client

      • getLoadBalancer

        public com.emc.rest.smart.LoadBalancer getLoadBalancer()

      • pingNode

        public PingResponse pingNode​(java.lang.String host)
        Description copied from interface: S3Client
        Issues an unauthenticated ping request to the specified host.
        Specified by:
        pingNode in interface S3Client

      • pingNode

        public PingResponse pingNode​(Protocol protocol,
                             java.lang.String host,
                             int port)
        Description copied from interface: S3Client
        Issues an unauthenticated ping request to the given host using the given protocol and port.
        Specified by:
        pingNode in interface S3Client

      • bucketExists

        public boolean bucketExists​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Returns whether bucketName exists in the user's namespace (or the configured namespace of the client). This call will return true if the bucket exists even if the user does not have access to the bucket. If this call returns false, a subsequent call to createBucket with the same name should succeed
        Specified by:
        bucketExists in interface S3Client

      • createBucket

        public void createBucket​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Creates a bucket with the specified name in the default namespace and with the default replication group
        Specified by:
        createBucket in interface S3Client

      • getBucketInfo

        public BucketInfo getBucketInfo​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Gets information about a bucket
        Specified by:
        getBucketInfo in interface S3Client

      • deleteBucket

        public void deleteBucket​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Deletes bucketName. The bucket must be empty of all objects and versions before it can be deleted
        Specified by:
        deleteBucket in interface S3Client

      • deleteBucket

        public void deleteBucket​(DeleteBucketRequest request)
        Description copied from interface: S3Client
        Deletes bucketName. The bucket could contain objects and versions before it can be deleted
        Specified by:
        deleteBucket in interface S3Client

      • setBucketAcl

        public void setBucketAcl​(java.lang.String bucketName,
                         CannedAcl cannedAcl)
        Description copied from interface: S3Client
        Sets the specified canned ACL on bucketName
        Specified by:
        setBucketAcl in interface S3Client
        See Also:
        CannedAcl

      • getBucketCors

        public CorsConfiguration getBucketCors​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Retrieves the CORS configuration for bucketName. If no CORS configuration exists for the specified bucket, null is returned
        Specified by:
        getBucketCors in interface S3Client
        See Also:
        CorsConfiguration

      • deleteBucketCors

        public void deleteBucketCors​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Removes the CORS configuration for bucketName
        Specified by:
        deleteBucketCors in interface S3Client

      • deleteBucketLifecycle

        public void deleteBucketLifecycle​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Deletes the lifecycle configuration for bucketName
        Specified by:
        deleteBucketLifecycle in interface S3Client

      • deleteBucketPolicy

        public void deleteBucketPolicy​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Deletes the bucket policy for bucketName
        Specified by:
        deleteBucketPolicy in interface S3Client
        See Also:
        BucketPolicy

      • getBucketLocation

        public LocationConstraint getBucketLocation​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Gets the location of bucketName. This call will return the name of the primary VDC of the bucket
        Specified by:
        getBucketLocation in interface S3Client

      • setBucketStaleReadAllowed

        public void setBucketStaleReadAllowed​(java.lang.String bucketName,
                                      boolean staleReadAllowed)
        Description copied from interface: S3Client
        Sets whether stale reads are allowed on bucketName. If true, during a temporary site outage (TSO), objects in the bucket may still be read from secondary sites, but these reads are not guaranteed to be strongly consistent (they may be stale if the primary site is inaccessible). Note that stale reads are not supported on filesystem buckets.
        Specified by:
        setBucketStaleReadAllowed in interface S3Client

      • listObjects

        public ListObjectsResult listObjects​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Lists all objects in bucketName with no restrictions
        Specified by:
        listObjects in interface S3Client

      • listObjects

        public ListObjectsResult listObjects​(java.lang.String bucketName,
                                     java.lang.String prefix)
        Description copied from interface: S3Client
        Lists objects in bucketName that start with prefix
        Specified by:
        listObjects in interface S3Client

      • listVersions

        public ListVersionsResult listVersions​(java.lang.String bucketName,
                                       java.lang.String prefix)
        Description copied from interface: S3Client
        Lists all versions of all objects in bucketName that start with prefix
        Specified by:
        listVersions in interface S3Client

      • putObject

        public void putObject​(java.lang.String bucketName,
                      java.lang.String key,
                      java.lang.Object content,
                      java.lang.String contentType)
        Description copied from interface: S3Client
        Creates or overwrites an object in bucketName named key containing content and having contentType
        Specified by:
        putObject in interface S3Client

      • putObject

        public void putObject​(java.lang.String bucketName,
                      java.lang.String key,
                      Range range,
                      java.lang.Object content)
        Description copied from interface: S3Client
        Updates object key in bucket bucketName at the specified byte range with new content
        Specified by:
        putObject in interface S3Client

      • appendObject

        public long appendObject​(java.lang.String bucketName,
                         java.lang.String key,
                         java.lang.Object content)
        Description copied from interface: S3Client
        Atomically appends to the end of object key in bucket bucketName with content and returns the starting offset of the append operation
        Specified by:
        appendObject in interface S3Client

      • copyObject

        public CopyObjectResult copyObject​(java.lang.String sourceBucketName,
                                   java.lang.String sourceKey,
                                   java.lang.String bucketName,
                                   java.lang.String key)
        Description copied from interface: S3Client
        Remotely copies object sourceKey in bucket sourceBucketName to key in bucketName
        Specified by:
        copyObject in interface S3Client

      • readObject

        public <T> T readObject​(java.lang.String bucketName,
                        java.lang.String key,
                        java.lang.Class<T> objectType)
        Description copied from interface: S3Client
        Reads object key in bucket bucketName and converts it to objectType, provided the conversion is supported by the implementation.

        Note: this method will return null for 304 and 412 responses (failed preconditions).

        Specified by:
        readObject in interface S3Client

      • readObject

        public <T> T readObject​(java.lang.String bucketName,
                        java.lang.String key,
                        java.lang.String versionId,
                        java.lang.Class<T> objectType)
        Description copied from interface: S3Client
        Reads version versionId of object key in bucket bucketName and converts it to objectType, provided the conversion is supported by the implementation.

        Note: this method will return null for 304 and 412 responses (failed preconditions).

        Specified by:
        readObject in interface S3Client

      • readObjectStream

        public java.io.InputStream readObjectStream​(java.lang.String bucketName,
                                            java.lang.String key,
                                            Range range)
        Description copied from interface: S3Client
        Reads range bytes of object key in bucket bucketName as a stream.

        Note: this method will return null for 304 and 412 responses (failed preconditions).

        Specified by:
        readObjectStream in interface S3Client

      • getObject

        public GetObjectResult<java.io.InputStream> getObject​(java.lang.String bucketName,
                                                      java.lang.String key)
        Description copied from interface: S3Client
        Gets object key in bucket bucketName. Object details as well as the data stream (obtained from GetObjectResult.getObject() are contained in the GetObjectResult instance.

        Note: this method will return null for 304 and 412 responses (failed preconditions). This method will open a stream for the object data. Be sure to call getObject() and, if requesting an InputStream, properly close the stream to release the connection.

        Specified by:
        getObject in interface S3Client

      • getObject

        public <T> GetObjectResult<T> getObject​(GetObjectRequest request,
                                        java.lang.Class<T> objectType)
        Description copied from interface: S3Client
        Gets an object using the parameters specified in request. Object details as well as the translated data (converted to objectType) are contained in the GetObjectResult instance.

        Note: this method will return null for 304 and 412 responses (failed preconditions). This method will open a stream for the object data. Be sure to call getObject() and, if requesting an InputStream, properly close the stream to release the connection.

        Specified by:
        getObject in interface S3Client

      • getPresignedUrl

        public java.net.URL getPresignedUrl​(java.lang.String bucketName,
                                    java.lang.String key,
                                    java.util.Date expirationTime)
        Description copied from interface: S3Client
        Generates a pre-signed URL to read object key in bucket bucketName. The URL will be valid until expirationTime
        Specified by:
        getPresignedUrl in interface S3Client

      • getPresignedUrl

        public java.net.URL getPresignedUrl​(PresignedUrlRequest request)
        Description copied from interface: S3Client
        Generates a pre-signed URL using the parameters specified in request
        Specified by:
        getPresignedUrl in interface S3Client

      • deleteObject

        public void deleteObject​(java.lang.String bucketName,
                         java.lang.String key)
        Description copied from interface: S3Client
        Deletes object key from bucket bucketName
        Specified by:
        deleteObject in interface S3Client

      • deleteVersion

        public void deleteVersion​(java.lang.String bucketName,
                          java.lang.String key,
                          java.lang.String versionId)
        Description copied from interface: S3Client
        Delets version versionId of object key in bucket bucketName.

        Note: versioning must be enabled in the bucket.

        Specified by:
        deleteVersion in interface S3Client

      • setObjectMetadata

        public void setObjectMetadata​(java.lang.String bucketName,
                              java.lang.String key,
                              S3ObjectMetadata objectMetadata)
        Description copied from interface: S3Client
        Sets metadata on object key in bucket bucketName
        Specified by:
        setObjectMetadata in interface S3Client

      • getObjectMetadata

        public S3ObjectMetadata getObjectMetadata​(java.lang.String bucketName,
                                          java.lang.String key)
        Description copied from interface: S3Client
        Gets metadata for object key in bucket bucketName
        Specified by:
        getObjectMetadata in interface S3Client

      • setObjectAcl

        public void setObjectAcl​(java.lang.String bucketName,
                         java.lang.String key,
                         CannedAcl cannedAcl)
        Specified by:
        setObjectAcl in interface S3Client

      • extendRetentionPeriod

        public void extendRetentionPeriod​(java.lang.String bucketName,
                                  java.lang.String key,
                                  java.lang.Long period)
        Description copied from interface: S3Client
        Extend retention period(seconds) on object key in bucket bucketName. Note: New retention period value can only be increased. That is, it can be the same as the current or greater value. If the new retention period value is -1, infinite retention applies on that object.
        Specified by:
        extendRetentionPeriod in interface S3Client

      • initiateMultipartUpload

        public java.lang.String initiateMultipartUpload​(java.lang.String bucketName,
                                                java.lang.String key)
        Specified by:
        initiateMultipartUpload in interface S3Client

      • listParts

        public ListPartsResult listParts​(java.lang.String bucketName,
                                 java.lang.String key,
                                 java.lang.String uploadId)
        Specified by:
        listParts in interface S3Client

      • setObjectLockConfiguration

        public void setObjectLockConfiguration​(java.lang.String bucketName,
                                       ObjectLockConfiguration objectLockConfiguration)
        Description copied from interface: S3Client
        Set Object Lock Configuration for bucket bucketName using parameters in objectLockConfiguration.
        Specified by:
        setObjectLockConfiguration in interface S3Client

      • getObjectLockConfiguration

        public ObjectLockConfiguration getObjectLockConfiguration​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Get the Object Lock configuration for bucket bucketName. If Object Lock Configuration is not set, null is returned.
        Specified by:
        getObjectLockConfiguration in interface S3Client

      • enableObjectLock

        public void enableObjectLock​(java.lang.String bucketName)
        Description copied from interface: S3Client
        Enable Object Lock for bucket bucketName.
        Specified by:
        enableObjectLock in interface S3Client

      • copyRange

        public CopyRangeResult copyRange​(CopyRangeRequest request)
        Description copied from interface: S3Client
        This API is an ECS extension API. IN addition to standard UploadPartCopy, it can source from multiple objects and reference ranges inside those objects. And it can work though with many Internet proxy servers, web servers (ECS included), and load balancers which may have a limit on HTTP headers.
        Specified by:
        copyRange in interface S3Client

      • getS3Config

        public S3Config getS3Config()