SQL Server 2008 to Oracle 11g SSIS data loading Performance…and Attunity!

Using the Native 32bit Drivers for Oracle within BIDS, i found that running a data load from SQL Server to Oracle was proving to be very slow – Transfer rate of 2350 rows per minute.

image

This was unacceptable as i had to data load a lot of data so as an experiment i tested OPENQUERY via linked server as from past experience this is generally quite quick at returning data.

I created a linked server within SQL Server using the OraOLEDB.Oracle provider and then tested a data load via a INSERT OPENQUERY:

--INSERT:
insert openquery([MYORACLESERVER.MY_ORACLE_SCHEMA], 
'SELECT 
my_id, 
my_effectivefrom, 
my_effectiveto, my_ref, 
my_imageid, 
my_imagenotexactmatch 
FROM MY_TABLE')
SELECT 
my_id, my_effectivefrom, my_effectiveto, my_ref, my_imageid, my_imagenotexactmatch 
FROM MY_TABLE

This did the data load within 6 minutes giving around a 5x increase in performance.

As a result i have now replaced my DATA FLOW tasks within SSIS with Execute SQL tasks:

image

This was good enough for what i was doing but i also came across the Attunity Connector for Oracle, so have been doing some testing with this connector.

Available here for SQL Server 2008 : http://www.microsoft.com/en-us/download/details.aspx?id=29284

Guide here: http://msdn.microsoft.com/en-us/library/ee470675(v=sql.100).aspx

More updates on this as I complete testing…

UPDATED – 2012-09-04 with Attunity Testing

The Attunity connector for Oracle has surpassed all my expectations by loaded the 87000 rows in jus over 2 seconds.

image

image

This shows that with the right drivers and tools and with some understanding of the underlying concepts, big gains can be made by tuning at the software level. My original table was likely to take 37 minutes and my proposed solution now takes 2.05 seconds! Approximately 1082x better

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: