Package org.intermine.objectstore.intermine

Class ObjectStoreWriterInterMineImpl

    • Field Detail

      • conn

        protected java.sql.Connection conn

      • connInUse

        protected boolean connInUse

      • batch

        protected Batch batch

      • createSituation

        protected java.lang.String createSituation

      • closeSituation

        protected java.lang.String closeSituation

      • recentSequences

        protected java.util.Map<java.lang.Integer,​java.lang.Boolean> recentSequences

      • tableToInfo

        protected java.util.Map<java.lang.String,​org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.TableInfo> tableToInfo

      • tableToColNameArray

        protected java.util.Map<java.lang.String,​java.lang.String[]> tableToColNameArray

      • tableToCollections

        protected java.util.Map<java.lang.String,​java.util.Set<CollectionDescriptor>> tableToCollections

      • connectionTakenBy

        protected java.lang.String connectionTakenBy

      • tablesAltered

        protected java.util.Set<java.lang.Object> tablesAltered

    • Constructor Detail

      • ObjectStoreWriterInterMineImpl

        public ObjectStoreWriterInterMineImpl​(ObjectStore os)
                               throws ObjectStoreException
        Constructor for this ObjectStoreWriter. This ObjectStoreWriter is bound to a single SQL Connection, grabbed from the provided ObjectStore.
        Parameters:
        os - an ObjectStoreInterMineImpl
        Throws:
        ObjectStoreException - if a problem occurs

    • Method Detail

      • execute

        public Results execute​(Query q,
                       int batchSize,
                       boolean optimise,
                       boolean explain,
                       boolean prefetch)
        Execute a Query on this ObjectStore
        Specified by:
        execute in interface ObjectStore
        Overrides:
        execute in class ObjectStoreInterMineImpl
        Parameters:
        q - the Query to execute
        batchSize - the batch size to initialise the Results object with
        optimise - whether to optimise queries
        explain - whether to explain queries
        prefetch - whether to use the PrefetchManager
        Returns:
        the results of the Query

      • executeSingleton

        public SingletonResults executeSingleton​(Query q,
                                         int batchSize,
                                         boolean optimise,
                                         boolean explain,
                                         boolean prefetch)
        Execute a Query on this ObjectStore, returning a SingletonResults
        Specified by:
        executeSingleton in interface ObjectStore
        Overrides:
        executeSingleton in class ObjectStoreInterMineImpl
        Parameters:
        q - the Query to execute
        batchSize - the batch size to initialise the Results object with
        optimise - whether to optimise queries
        explain - whether to explain queries
        prefetch - whether to use the PrefetchManager
        Returns:
        the results of the Query

      • everOptimise

        public boolean everOptimise()
        Returns whether optimisation should be permitted. For the ObjectStore, this will always be true, but for the ObjectStoreWriter, it may be false if there is written data that has not been committed yet.
        Overrides:
        everOptimise in class ObjectStoreInterMineImpl
        Returns:
        a boolean

      • getLog

        public java.io.Writer getLog()
        Returns the log used by this objectstore.
        Overrides:
        getLog in class ObjectStoreInterMineImpl
        Returns:
        the log

      • setLog

        public void setLog​(java.io.Writer log)
        Not implemented.
        Overrides:
        setLog in class ObjectStoreInterMineImpl
        Parameters:
        log - ignored

      • dbLog

        protected void dbLog​(long optimise,
                     long estimated,
                     long execute,
                     long permitted,
                     long convert,
                     Query q,
                     java.lang.String sql)
        Produce an entry in the DB log.
        Overrides:
        dbLog in class ObjectStoreInterMineImpl
        Parameters:
        optimise - the number of milliseconds used to optimise the query
        estimated - the estimated number of milliseconds required to run the query
        execute - the number of milliseconds spent executing the query
        permitted - an acceptable number of milliseconds for the query to take
        convert - the number of milliseconds spent converting the results
        q - the Query run
        sql - the SQL string executed

      • setLogEverything

        public void setLogEverything​(boolean logEverything)
        Sets the logEverything configuration option.
        Overrides:
        setLogEverything in class ObjectStoreInterMineImpl
        Parameters:
        logEverything - a boolean

      • setVerboseQueryLog

        public void setVerboseQueryLog​(boolean verboseQueryLog)
        Sets the verboseQueryLog configuration option.
        Overrides:
        setVerboseQueryLog in class ObjectStoreInterMineImpl
        Parameters:
        verboseQueryLog - a boolean

      • setLogExplains

        public void setLogExplains​(boolean logExplains)
        Sets the logExplains configuration option.
        Overrides:
        setLogExplains in class ObjectStoreInterMineImpl
        Parameters:
        logExplains - a boolean

      • setLogBeforeExecute

        public void setLogBeforeExecute​(boolean logBeforeExecute)
        Sets the logBeforeExecute configuration option.
        Overrides:
        setLogBeforeExecute in class ObjectStoreInterMineImpl
        Parameters:
        logBeforeExecute - a boolean

      • setDisableResultsCache

        public void setDisableResultsCache​(boolean disableResultsCache)
        Sets the disableResultsCache configuration option.
        Overrides:
        setDisableResultsCache in class ObjectStoreInterMineImpl
        Parameters:
        disableResultsCache - a boolean

      • setBatchWriter

        public void setBatchWriter​(BatchWriter batchWriter)
                    throws ObjectStoreException
        Allows the changing of the BatchWriter that this ObjectStoreWriter uses.
        Parameters:
        batchWriter - the new BatchWriter - use BatchWriterSimpleImpl for writers likely to see small batches, and optimised (eg BatchWriterPostgresCopyImpl) implementations for bulk-write writers.
        Throws:
        ObjectStoreException - if something goes wrong

      • setMinBagTableSize

        public void setMinBagTableSize​(int minBagTableSize)
        Set the cutoff value used to decide if a bag should be put in a table.
        Overrides:
        setMinBagTableSize in class ObjectStoreInterMineImpl
        Parameters:
        minBagTableSize - don't use a table to represent bags if the bag is smaller than this value

      • getMinBagTableSize

        public int getMinBagTableSize()
        Returns the cutoff value used to decide if a bag should be put in a table.
        Overrides:
        getMinBagTableSize in class ObjectStoreInterMineImpl
        Returns:
        an int

      • getConnection

        public java.sql.Connection getConnection()
                                  throws java.sql.SQLException
        Returns a Connection. Please put them back. Whenever you receive a connection from the object-store, you MUST release its resources by calling releaseConnection. Failure to do so KILLS THE OBJECT STORE!
        Overrides:
        getConnection in class ObjectStoreInterMineImpl
        Returns:
        a java.sql.Connection
        Throws:
        java.sql.SQLException - if there is a problem with that

      • releaseConnection

        public void releaseConnection​(java.sql.Connection c)
        Allows one to put a connection back.
        Overrides:
        releaseConnection in class ObjectStoreInterMineImpl
        Parameters:
        c - a Connection

      • doFinalise

        protected void doFinalise()
        Overrides Object.finalize - release the connection back to the objectstore.
        Overrides:
        doFinalise in class ObjectStoreInterMineImpl

      • getObjectStore

        public ObjectStore getObjectStore()
        Retrieve this Writer's ObjectStore. This ObjectStoreWriter is a read-write extension to the read-only ObjectStore. This ObjectStoreWriter uses a single database connection in order to allow proper transaction support - use the ObjectStore for general read-only access.
        Specified by:
        getObjectStore in interface ObjectStoreWriter
        Returns:
        the ObjectStore

      • store

        public void store​(java.lang.Object o)
           throws ObjectStoreException
        Store an object in this ObjectStore. If the ID of this object is not set, it will be set to a previously unused value. If the ID matches the ID of an object already in the objectstore, then an update operation is performed - otherwise a store operation is performed.
        Attributes and references of the stored object will be set to those of the provided object. Collections of the stored object will contain the union of the contents of any pre-existing stored object, and the contents of the provided object. There is currently no way to remove entries from a collection of a stored object.
        Any objects referred to by this object will have their ID set in a similar way to this object. This object will be stored with references and collections using those IDs, however those objects will not be stored themselves. Therefore, the objectstore will be inconsistent until those objects are also stored.
        If bidirectional relationships are not consistent in the objects being stored, then the behaviour of the store method is not defined. Specifically, one-to-one relationships must be stored on both sides. For example, if A has a relationship with B1 in the database, and we store a new A that has a relationship with B2, then we must also store B2 with a relationship back to A (to complete the relationship), and store B1 (to completely break the old relationship),
        Specified by:
        store in interface ObjectStoreWriter
        Parameters:
        o - the object to store
        Throws:
        ObjectStoreException - if an error occurs during storage of the object

      • storeWithConnection

        protected void storeWithConnection​(java.sql.Connection c,
                                   java.lang.Object o)
                            throws ObjectStoreException
        Performs store with a given Connection.
        Parameters:
        c - the Connection
        o - the object to store
        Throws:
        ObjectStoreException - sometimes

      • addToCollection

        public void addToCollection​(java.lang.Integer hasId,
                            java.lang.Class<?> clazz,
                            java.lang.String fieldName,
                            java.lang.Integer hadId)
                     throws ObjectStoreException
        Place an object in a many-to-many collection of another object. This method provides a way to place an object into a collection in another object without having either object loaded in memory. Only the IDs of the two objects are required, along with the name of the collection.
        Specified by:
        addToCollection in interface ObjectStoreWriter
        Parameters:
        hasId - the ID of the object that has the collection
        clazz - the class of the object that has the collection
        fieldName - the name of the collection
        hadId - the ID of the object to place in the collection
        Throws:
        ObjectStoreException - if a problem occurs

      • addToCollectionWithConnection

        protected void addToCollectionWithConnection​(java.sql.Connection c,
                                             java.lang.Integer hasId,
                                             java.lang.Class<?> clazz,
                                             java.lang.String fieldName,
                                             java.lang.Integer hadId)
                                      throws ObjectStoreException
        Performs addToCollection with given connection
        Parameters:
        c - the Connection
        hasId - the ID of the object that has the collection
        clazz - the class of the object that has the collection
        fieldName - the name of the collection
        hadId - the ID of the object to place in the collection
        Throws:
        ObjectStoreException - if an error occurs

      • getTableInfo

        protected org.intermine.objectstore.intermine.ObjectStoreWriterInterMineImpl.TableInfo getTableInfo​(ClassDescriptor tableMaster)
                                                                                             throws ObjectStoreException
        Produces metadata for a given table, caching it to save time.
        Parameters:
        tableMaster - the ClassDescriptor describing the table
        Returns:
        a TableInfo object
        Throws:
        ObjectStoreException - if something goes wrong

      • populateIds

        protected boolean populateIds​(java.sql.Connection c,
                              InterMineObject o)
                       throws java.sql.SQLException,
                              java.lang.IllegalAccessException
        Populates the object o with IDs.
        Parameters:
        c - a Connection with which to fetch more IDs
        o - the InterMineObject
        Returns:
        true if the object will need to be deleted from the DB before a store
        Throws:
        java.sql.SQLException - if the database cannot produce a new ID
        java.lang.IllegalAccessException - if the ID field cannot be set

      • writePendingClobs

        protected void writePendingClobs​(java.sql.Connection c,
                                 java.lang.Object o)
                          throws ObjectStoreException,
                                 java.sql.SQLException,
                                 java.lang.IllegalAccessException
        Writes the contents of any pending Clobs to the database, and replaces them in the objects with a real ClobAccess object.
        Parameters:
        c - a connection
        o - the object to transform
        Throws:
        ObjectStoreException - if something goes wrong
        java.sql.SQLException - if something goes wrong
        java.lang.IllegalAccessException - if something goes wrong

      • addAllToBagWithConnection

        protected void addAllToBagWithConnection​(java.sql.Connection c,
                                         ObjectStoreBag osb,
                                         java.util.Collection<java.lang.Integer> coll)
                                  throws ObjectStoreException
        Adds elements to the given bag.
        Parameters:
        c - a Connection
        osb - an ObjectStoreBag
        coll - a Collection of Integers
        Throws:
        ObjectStoreException - if there is an error in the underlying database

      • removeAllFromBagWithConnection

        protected void removeAllFromBagWithConnection​(java.sql.Connection c,
                                              ObjectStoreBag osb,
                                              java.util.Collection<java.lang.Integer> coll)
                                       throws ObjectStoreException
        Removes elements from the given bag.
        Parameters:
        c - a Connection
        osb - an ObjectStoreBag
        coll - a Collection of Integers
        Throws:
        ObjectStoreException - if there is an error in the underlying database

      • addToBagFromQuery

        public void addToBagFromQuery​(ObjectStoreBag osb,
                              Query query)
                       throws ObjectStoreException
        Adds elements to an ObjectStoreBag from the results of a Query. The data may not be loaded into Java, so this is a performance improvement method. For example, in SQL this method may issue a command like "INSERT INTO bag SELECT ...".
        Specified by:
        addToBagFromQuery in interface ObjectStoreWriter
        Parameters:
        osb - an ObjectStoreBag
        query - an objectstore Query, which contains only one result column, which is a suitable type for insertion into the given bag
        Throws:
        ObjectStoreException - if something goes wrong

      • addToBagFromQueryWithConnection

        protected void addToBagFromQueryWithConnection​(java.sql.Connection c,
                                               ObjectStoreBag osb,
                                               Query query)
                                        throws ObjectStoreException
        Adds elements to a bag from the results of a query.
        Parameters:
        c - a Connection
        osb - an ObjectStoreBag
        query - a Query with only one column
        Throws:
        ObjectStoreException - if there is an error in the underlying database

      • replaceClob

        public void replaceClob​(Clob clob,
                        java.lang.String text)
                 throws ObjectStoreException
        Replaces the contents of the given Clob with the given String.
        Specified by:
        replaceClob in interface ObjectStoreWriter
        Parameters:
        clob - the Clob to write to
        text - the text to write to the Clob
        Throws:
        ObjectStoreException - if something goes wrong

      • replaceClobWithConnection

        public void replaceClobWithConnection​(java.sql.Connection c,
                                      Clob clob,
                                      java.lang.String text)
                               throws ObjectStoreException
        Replaces the contents of the given Clob with the given String.
        Parameters:
        c - a Connection
        clob - the Clob to write to
        text - the text to write to the Clob
        Throws:
        ObjectStoreException - if something goes wrong

      • delete

        public void delete​(QueryClass qc,
                   Constraint c)
            throws ObjectStoreException
        Deletes a set of objects from this ObjectStore.
        Specified by:
        delete in interface ObjectStoreWriter
        Parameters:
        qc - a QueryClass for the class of objects to delete. The class given in the query class must not inherit from org.intermine.model.InterMineObject.
        c - a Constraint based on the QueryClass to filter the objects to delete, or null to delete all objects
        Throws:
        ObjectStoreException - if an error occurs while deleting the objects

      • deleteWithConnection

        public void deleteWithConnection​(java.sql.Connection con,
                                 QueryClass qc,
                                 Constraint c)
                          throws ObjectStoreException
        Performs a delete, with a connection.
        Parameters:
        con - the Connection
        qc - the QueryClass in which to delete - note that this must currently be a Simple Object class
        c - the Constraint to limit the deletes, or null to delete everything
        Throws:
        ObjectStoreException - if something goes wrong

      • isInTransactionWithConnection

        protected boolean isInTransactionWithConnection​(java.sql.Connection c)
                                         throws ObjectStoreException
        Finds if we are in a transaction.
        Parameters:
        c - the Connection
        Returns:
        true or false
        Throws:
        ObjectStoreException - sometimes

      • beginTransactionWithConnection

        protected void beginTransactionWithConnection​(java.sql.Connection c)
                                       throws ObjectStoreException
        Begins a transaction.
        Parameters:
        c - the Connection
        Throws:
        ObjectStoreException - if we are already in a transaction

      • commitTransactionWithConnection

        protected void commitTransactionWithConnection​(java.sql.Connection c)
                                        throws ObjectStoreException
        Commits a transaction.
        Parameters:
        c - the Connection
        Throws:
        ObjectStoreException - if we are not in a transaction

      • abortTransactionWithConnection

        public void abortTransactionWithConnection​(java.sql.Connection c)
                                    throws ObjectStoreException
        Aborts a transaction.
        Parameters:
        c - the Connection
        Throws:
        ObjectStoreException - if we are not in a transaction

      • batchCommitTransaction

        public void batchCommitTransaction()
                            throws ObjectStoreException
        Request that the ObjectStoreWriter commits and closes the transaction and then opens a new one, without guaranteeing that the operation is finished before this method returns. Note that writes that are made AFTER this method may be treated as if they were BEFORE this method and be committed in the transaction.
        Specified by:
        batchCommitTransaction in interface ObjectStoreWriter
        Throws:
        ObjectStoreException - if an error occurs

      • batchCommitTransactionWithConnection

        public void batchCommitTransactionWithConnection​(java.sql.Connection c)
                                          throws ObjectStoreException
        Commits a transaction and opens a new one, without guaranteeing the operation is finished before this method returns.
        Parameters:
        c - the Connection
        Throws:
        ObjectStoreException - if an error occurs

      • databaseAltered

        public void databaseAltered​(java.util.Set<java.lang.Object> tablesChanged)
        Internal method called by the ObjectStoreWriter, to notify the ObjectStore that some of the data in the database has changed. This method should never be called on an ObjectStoreWriter.
        Overrides:
        databaseAltered in class ObjectStoreInterMineImpl
        Parameters:
        tablesChanged - a Set of table names that may have been altered

      • getSequence

        public java.util.Map<java.lang.Object,​java.lang.Integer> getSequence​(java.util.Set<java.lang.Object> tables)
        Returns an object representing the current state of the database, for fail-fast concurrency control. Delegate to the parent ObjectStore.
        Specified by:
        getSequence in interface ObjectStore
        Overrides:
        getSequence in class ObjectStoreAbstractImpl
        Parameters:
        tables - a Set of objects representing independent components of the database
        Returns:
        a Map containing sequence data

      • checkSequence

        public void checkSequence​(java.util.Map<java.lang.Object,​java.lang.Integer> sequence,
                          Query q,
                          java.lang.String message)
                   throws DataChangedException
        Checks a number against the sequence number, and throws an exception if they do not match. Delegate to the parent ObjectStore.
        Overrides:
        checkSequence in class ObjectStoreAbstractImpl
        Parameters:
        sequence - a Map representing a database state
        q - the Query that is to be run
        message - some description of the operation that is about to happen
        Throws:
        DataChangedException - if the sequence numbers do not match

      • execute

        public java.util.List<ResultsRow<java.lang.Object>> execute​(Query q,
                                                            int start,
                                                            int limit,
                                                            boolean optimise,
                                                            boolean explain,
                                                            java.util.Map<java.lang.Object,​java.lang.Integer> sequence)
                                                     throws ObjectStoreException
        Execute a Query on this ObjectStore, asking for a certain range of rows to be returned. This will usually only be called by the Results object returned from execute(Query q). This method is overridden in order to flush batches properly before the read.
        Specified by:
        execute in interface ObjectStore
        Overrides:
        execute in class ObjectStoreInterMineImpl
        Parameters:
        q - the Query to execute
        start - the start row
        limit - the maximum number of rows to return
        optimise - true if it is expected that optimising the query will improve performance
        explain - true if the ObjectStore should enforce maximum query running time constraints
        sequence - an object representing the state of the database corresponding to when the action that resulted in this execute was started. This number must match the ObjectStore's internal sequence number or a DataChangedException is thrown. The sequence number is incremented each time the data in the objectstore is changed
        Returns:
        a List of ResultRows
        Throws:
        ObjectStoreException - if an error occurs during the running of the Query

      • generateSql

        protected java.lang.String generateSql​(java.sql.Connection c,
                                       Query q,
                                       int start,
                                       int limit)
                                throws ObjectStoreException
        Create temporary tables for the bag in the BagConstraints of the given Query, then call SqlGenerator.generate(). Delegates to the parent ObjectStore
        Overrides:
        generateSql in class ObjectStoreInterMineImpl
        Parameters:
        c - a Connection to use
        q - the Query
        start - the start row number (inclusive, from zero)
        limit - maximum number of rows to return
        Returns:
        the SQL for the Query
        Throws:
        ObjectStoreException - if an error occurs

      • count

        public int count​(Query q,
                 java.util.Map<java.lang.Object,​java.lang.Integer> sequence)
          throws ObjectStoreException
        Counts the number of rows the query will produce This method is overridden in order to flush batches properly before the read.
        Specified by:
        count in interface ObjectStore
        Overrides:
        count in class ObjectStoreInterMineImpl
        Parameters:
        q - InterMine Query on which to count rows
        sequence - an object representing the state of the database corresponding to when the action that resulted in this execute was started. This number must match the ObjectStore's internal sequence number or a DataChangedException is thrown. The sequence number is incremented each time the data in the objectstore is changed
        Returns:
        the number of rows that will be produced by query
        Throws:
        ObjectStoreException - if an error occurs counting the query

      • internalGetObjectById

        protected InterMineObject internalGetObjectById​(java.lang.Integer id,
                                                java.lang.Class<? extends InterMineObject> clazz)
                                         throws ObjectStoreException
        Internal service method for getObjectById. This method is overridden in order to improve the performance of the operation - this implementation does not bother with the EXPLAIN call to the underlying SQL database. This method is overridden in order to flush matches properly before the read.
        Overrides:
        internalGetObjectById in class ObjectStoreInterMineImpl
        Parameters:
        id - the ID of the object to get
        clazz - a class of the object
        Returns:
        an object from the database
        Throws:
        ObjectStoreException - if an error occurs during the running of the Query

      • isMultiConnection

        public boolean isMultiConnection()
        Return whether or not this ObjectStore gives a performance improvement when multiple simultaneous are made. Note that ALL Objectstore must be multi-threading safe. If this method returns true, then the ObjectStore probably handles multiple connections to the database. The Results class uses this to work out whether or not to do prefetching.
        Specified by:
        isMultiConnection in interface ObjectStore
        Overrides:
        isMultiConnection in class ObjectStoreInterMineImpl
        Returns:
        true if one should do multiple simultaneous operations

      • getSerialWithConnection

        protected java.lang.Integer getSerialWithConnection​(java.sql.Connection c)
                                             throws java.sql.SQLException
        Overrides, in order to store recentSequences.
        Overrides:
        getSerialWithConnection in class ObjectStoreInterMineImpl
        Parameters:
        c - the Connection
        Returns:
        an Integer
        Throws:
        java.sql.SQLException - if an error occurs