Showing posts with label bcp. Show all posts
Showing posts with label bcp. Show all posts

Thursday, March 8, 2012

"large value" data type(bcp)

I want to store some binary things(pic and so on), so I create a table which contain a a "varbinary" data-type column.

but 1. I used OPENROWSET to insert the large file in this table. 2. I used master..xp_cmdshell to retrieve data out as a file. One strange thing happened: the size of the input and output is really different(output is 1k bigger than the input file).

and it seems that the file is broken with different file format......

I really don't know why....

Any help would be appriciated.....

kavin

Could you please post a simple repro script? What command or utility are you using with xp_cmdshell to create the file? Is it BCP or OSQL with SELECT or BCP with queryout option and so on?|||

yes, sure.

EXEC master..xp_cmdshell 'bcp " SELECT column1 FROM Products where id =1111111117 " queryout fileName -n -U sa -P -S yourserver.

Is it any problem?

Anyway, thanks.

BR,

kavin

|||and when I compare the two files, I found that it just the 8-bytes at the beginning of file are extra filled. So I'm afraid something wrong with the created file?

Sunday, February 19, 2012

"Additional information: System error" message

Hello,
I needed to look at some .BCP files, so I thought I'd create a small tool for that.
I decided to create a temporary
DB, attach empty mdf, ldf files with predefined structure and then insert the BCP files, so I can view the data as DB tables .

Here is the procedure I follow:
1) I drop the existing (if exists) DB ("Drop DBName") - to be prepared for the next set of .BCP files.
2) Then I create a new
DB using sp_attach_db,
3) Then I populate the db via "Bulk Insert".

1st time this works just fine, but when I try to execute step 1 (drop DB) after step 3 I get a message that there is still open connection and the DB can't be dropped.

Here I can do two different things:
1) Find the
SPID of the open connection to my DB and KILL the process or
2) Instead of killing the process, set the DB to "SINGLE_USER WITH ROLLBACK IMMEDIATE" and then drop the DB.

In both cases I was able to Drop the DB and attach the new one, however the Bulk Insert (via ExecuteNonQuery() ) for some reason closes the current connection and brakes with the "System Error" message.

I simulated this procedure on Query Analyser and on SQL Server Studio Express, but everything worked just fine there.

The VB code is a little complicated with using app.config to open so many network drives and folders, but if anybody is interested I'll post it.

To make it short, here is the SQL Query procedure (which worked, though):

USE Master
if exists
(select name
from master.dbo.sysdatabases
where has_dbaccess(name)=1 and name='MyDB')
BEGIN
Drop database MyDB
END
exec sp_who2

-- Before the next command I have to manually restore the .ndf, .ldf files, deleted by the "Drop DB" command

sp_attach_db @.dbname = N'MyDB',
@.filename1 = N'D:\CurrSchema\DB_Patient.ndf',
@.filename2 = N'D:\CurrSchema\DB_Definitions.mdf',
@.filename3 = N'D:\CurrSchema\DB_Log.ldf',
@.filename4 = N'D:\CurrSchema\DB_Data.ndf'
exec sp_who2

-- WAITFOR DELAY '00:00:04'
-- need to wait manually here. Waitfor won't work with "USE DB"

USE MyDB
BULK INSERT CAL
FROM '\\srv\D$\CurrFiles\CAL.bcp'
WITH (DATAFILETYPE='native',KEEPIDENTITY,KEEPNULLS)

BULK INSERT CAP
FROM '\\srv\CurrFiles\CAP.bcp'
WITH (DATAFILETYPE='native',KEEPIDENTITY,KEEPNULLS)
exec sp_who2

Has anybody seen something like this?

Thank you.

Haven't seen it personally. However, I suggest you insert a 'go' between each command to seperate them into batches. Also, be sure to change the database to 'master' for your connection before sending/executing 'drop db'.|||

Yes, Thank you - you were right. I did not need the 'GO', but the change to 'master' was the key. I should have been more careful when using somebody else's code - no matter how encapsulated, it's never context-free - in my case I had to add context switching with 'USE master'.

George.

|||

Hi... you mentioned you had the below source sample...

The VB code is a little complicated with using app.config to open so many network drives and folders, but if anybody is interested I'll post it.

Can you please post or sent to billbbellevue@.hotmail.com

Kind regards,

Bill Bezick

"Additional information: System error" message

Hello,
I needed to look at some .BCP files, so I thought I'd create a small tool for that.
I decided to create a temporary
DB, attach empty mdf, ldf files with predefined structure and then insert the BCP files, so I can view the data as DB tables .

Here is the procedure I follow:
1) I drop the existing (if exists) DB ("Drop DBName") - to be prepared for the next set of .BCP files.
2) Then I create a new
DB using sp_attach_db,
3) Then I populate the db via "Bulk Insert".

1st time this works just fine, but when I try to execute step 1 (drop DB) after step 3 I get a message that there is still open connection and the DB can't be dropped.

Here I can do two different things:
1) Find the
SPID of the open connection to my DB and KILL the process or
2) Instead of killing the process, set the DB to "SINGLE_USER WITH ROLLBACK IMMEDIATE" and then drop the DB.

In both cases I was able to Drop the DB and attach the new one, however the Bulk Insert (via ExecuteNonQuery() ) for some reason closes the current connection and brakes with the "System Error" message.

I simulated this procedure on Query Analyser and on SQL Server Studio Express, but everything worked just fine there.

The VB code is a little complicated with using app.config to open so many network drives and folders, but if anybody is interested I'll post it.

To make it short, here is the SQL Query procedure (which worked, though):

USE Master
if exists
(select name
from master.dbo.sysdatabases
where has_dbaccess(name)=1 and name='MyDB')
BEGIN
Drop database MyDB
END
exec sp_who2

-- Before the next command I have to manually restore the .ndf, .ldf files, deleted by the "Drop DB" command

sp_attach_db @.dbname = N'MyDB',
@.filename1 = N'D:\CurrSchema\DB_Patient.ndf',
@.filename2 = N'D:\CurrSchema\DB_Definitions.mdf',
@.filename3 = N'D:\CurrSchema\DB_Log.ldf',
@.filename4 = N'D:\CurrSchema\DB_Data.ndf'
exec sp_who2

-- WAITFOR DELAY '00:00:04'
-- need to wait manually here. Waitfor won't work with "USE DB"

USE MyDB
BULK INSERT CAL
FROM '\\srv\D$\CurrFiles\CAL.bcp'
WITH (DATAFILETYPE='native',KEEPIDENTITY,KEEPNULLS)

BULK INSERT CAP
FROM '\\srv\CurrFiles\CAP.bcp'
WITH (DATAFILETYPE='native',KEEPIDENTITY,KEEPNULLS)
exec sp_who2

Has anybody seen something like this?

Thank you.

Haven't seen it personally. However, I suggest you insert a 'go' between each command to seperate them into batches. Also, be sure to change the database to 'master' for your connection before sending/executing 'drop db'.|||

Yes, Thank you - you were right. I did not need the 'GO', but the change to 'master' was the key. I should have been more careful when using somebody else's code - no matter how encapsulated, it's never context-free - in my case I had to add context switching with 'USE master'.

George.

|||

Hi... you mentioned you had the below source sample...

The VB code is a little complicated with using app.config to open so many network drives and folders, but if anybody is interested I'll post it.

Can you please post or sent to billbbellevue@.hotmail.com

Kind regards,

Bill Bezick

Thursday, February 9, 2012

#@ Row 1, Column 7: String data, right truncation @#

I am trying to BCP a ton of data files into a SQL 2005 database. The first row of data in one of my files looks like:

1000|100000156752|100000176409|100000000000|100000000000|9.4|M|9.4||1/22/1993||1||||100|||||||||1|1/22/1993|||||||

The error file has this:

#@. Row 1, Column 7: String data, right truncation @.#
0 0 0 0 0 .00 17|27.7|M|27.7||2/2/1993||1||||100|||||||||1|2/2/1993|||||||

I've built a format file based on the SQL Table definitions with this command:
FOR %%f IN (*.*) DO bcp IRIS.dbo.%%f format nul -T -n -t"|" -r"\n" -f%%f.format

The format file for lines 1-8 look like:
1 SQLNUMERIC 1 19 "|" 1 SITE_ID ""
2 SQLNUMERIC 1 19 "|" 2 WEL_ID ""
3 SQLNUMERIC 1 19 "|" 3 WPOO_ID ""
4 SQLNUMERIC 1 19 "|" 4 WSMP_ID ""
5 SQLNUMERIC 1 19 "|" 5 CC_ID ""
6 SQLNUMERIC 1 19 "|" 6 CORE_LENGTH ""
7 SQLCHAR 2 2 "|" 7 LENGTH_MEASM_UNIT_ID SQL_Latin1_General_CP1_CI_AS
8 SQLNUMERIC 1 19 "|" 8 LENGTH_OF_CORE_RECOVD ""

Here is the DOS command window results for this file:
F:\Data>cd import

F:\Data\Import>FOR %f IN (CONVT_CORES.*) DO bcp IRIS.dbo.%f in %f -e..\BCP_Error
\%f.error -Slocalhost -Usa -Psol3admin -f..\BCP_Format\%f.format

F:\Data\Import>bcp IRIS.dbo.CONVT_CORES in CONVT_CORES -e..\BCP_Error\CONVT_CORE
S.error -Slocalhost -Usa -Psol3admin -f..\BCP_Format\CONVT_CORES.format

Starting copy...
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22003, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]Numeric value out of range
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation

BCP copy in failed

F:\Data\Import>cd..

So, what is wrong?

Seems, that the data types do not make, make sure that the data types in the destination tables are capable to import the data, otherwsie truncation will take place and will chop the data.

HTH, Jens K. Suessmeyer.


http://www.sqlserver2005.de

|||All datat types are made larger than needed by the largest data piece being imported. It works quite easily in SSIS but BCP gives me this error. With a few hundred tables, SSIS will take me days to setup and execute each one.

Something is wrong with BCP in SQL 2005!!!|||

Hi Guys,

Make sure the table schema on the target server matches the one on the source server. This will solve the proble. There is nothing wrong in the BCP in SQL 2005.

Roshan.

|||

Try using sqlchar across the board. If you are exporting from a SQL server and then importing the data back in, use native format to export, and specify this in your format file.

I ran into this problem myself a few weeks ago.

|||Exporting from Oracle. I've even gone so far as to make all columns varchar(xx) to see if that would fix it. It does *not* fix it. I've tried various formats for the CSV and they all give me this error. I exported to XML, wrote a small app to import that and I have problems with one, and only one, table out of 736 tables. I cannot say BCP works as I cannot get it to work. SSIS works on all but 7 tables so I know the data can go in. <bah type="humbug" />|||I have bcp jobs that run every night from production to staging, and one started failing after putting SP2 on SQL server. It was only on one table. I finally fixed the problem by deleting the table and recreating it from a script extracted from production. All I can figure is that something in the table schema got corrupted.

#@ Row 1, Column 7: String data, right truncation @#

I am trying to BCP a ton of data files into a SQL 2005 database. The first row of data in one of my files looks like:

1000|100000156752|100000176409|100000000000|100000000000|9.4|M|9.4||1/22/1993||1||||100|||||||||1|1/22/1993|||||||

The error file has this:

#@. Row 1, Column 7: String data, right truncation @.#
0 0 0 0 0 .00 17|27.7|M|27.7||2/2/1993||1||||100|||||||||1|2/2/1993|||||||

I've built a format file based on the SQL Table definitions with this command:
FOR %%f IN (*.*) DO bcp IRIS.dbo.%%f format nul -T -n -t"|" -r"\n" -f%%f.format

The format file for lines 1-8 look like:
1 SQLNUMERIC 1 19 "|" 1 SITE_ID ""
2 SQLNUMERIC 1 19 "|" 2 WEL_ID ""
3 SQLNUMERIC 1 19 "|" 3 WPOO_ID ""
4 SQLNUMERIC 1 19 "|" 4 WSMP_ID ""
5 SQLNUMERIC 1 19 "|" 5 CC_ID ""
6 SQLNUMERIC 1 19 "|" 6 CORE_LENGTH ""
7 SQLCHAR 2 2 "|" 7 LENGTH_MEASM_UNIT_ID SQL_Latin1_General_CP1_CI_AS
8 SQLNUMERIC 1 19 "|" 8 LENGTH_OF_CORE_RECOVD ""

Here is the DOS command window results for this file:
F:\Data>cd import

F:\Data\Import>FOR %f IN (CONVT_CORES.*) DO bcp IRIS.dbo.%f in %f -e..\BCP_Error
\%f.error -Slocalhost -Usa -Psol3admin -f..\BCP_Format\%f.format

F:\Data\Import>bcp IRIS.dbo.CONVT_CORES in CONVT_CORES -e..\BCP_Error\CONVT_CORE
S.error -Slocalhost -Usa -Psol3admin -f..\BCP_Format\CONVT_CORES.format

Starting copy...
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22003, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]Numeric value out of range
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation
SQLState = 22001, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]String data, right truncation

BCP copy in failed

F:\Data\Import>cd..

So, what is wrong?

Seems, that the data types do not make, make sure that the data types in the destination tables are capable to import the data, otherwsie truncation will take place and will chop the data.

HTH, Jens K. Suessmeyer.


http://www.sqlserver2005.de

|||All datat types are made larger than needed by the largest data piece being imported. It works quite easily in SSIS but BCP gives me this error. With a few hundred tables, SSIS will take me days to setup and execute each one.

Something is wrong with BCP in SQL 2005!!!|||

Hi Guys,

Make sure the table schema on the target server matches the one on the source server. This will solve the proble. There is nothing wrong in the BCP in SQL 2005.

Roshan.

|||

Try using sqlchar across the board. If you are exporting from a SQL server and then importing the data back in, use native format to export, and specify this in your format file.

I ran into this problem myself a few weeks ago.

|||Exporting from Oracle. I've even gone so far as to make all columns varchar(xx) to see if that would fix it. It does *not* fix it. I've tried various formats for the CSV and they all give me this error. I exported to XML, wrote a small app to import that and I have problems with one, and only one, table out of 736 tables. I cannot say BCP works as I cannot get it to work. SSIS works on all but 7 tables so I know the data can go in. <bah type="humbug" />|||I have bcp jobs that run every night from production to staging, and one started failing after putting SP2 on SQL server. It was only on one table. I finally fixed the problem by deleting the table and recreating it from a script extracted from production. All I can figure is that something in the table schema got corrupted.

##tblTemp invisible for bcp

I did create #tblTemp on sql but can not use it with "outside" bcp routine
from command line:
Error = [Microsoft][ODBC SQL ...][SQL Server]Invalid object name '##tblTemp'.
Any idea why?
--
gokWhy do you want to BCP into a global temp table? If you are already in tsql
then why not use Bulk Insert instead? Not that I am recommending you use
temp tables but at least Bulk Insert can see a local temp table.
Andrew J. Kelly SQL MVP
"gok" <gok@.discussions.microsoft.com> wrote in message
news:A182DF5D-B64F-4E66-8F38-7A06EEFD6159@.microsoft.com...
>I did create #tblTemp on sql but can not use it with "outside" bcp routine
> from command line:
> Error = [Microsoft][ODBC SQL ...][SQL Server]Invalid object name
> '##tblTemp'.
> Any idea why?
> --
> gok
>|||I have a data file (from user verification in xsl) I need to import to sql.
the idea was to give client copy of bcp.exe and he will post data from his
machine to sql. In BULK INSERT I have to use shared folder to import data
from file, otherwise this file still "invisible" for sql (I was trying query
analyzer on non-sql machine).
What would be a correct way to append data from file?
"Andrew J. Kelly" wrote:

> Why do you want to BCP into a global temp table? If you are already in ts
ql
> then why not use Bulk Insert instead? Not that I am recommending you use
> temp tables but at least Bulk Insert can see a local temp table.
> --
> Andrew J. Kelly SQL MVP
>
> "gok" <gok@.discussions.microsoft.com> wrote in message
> news:A182DF5D-B64F-4E66-8F38-7A06EEFD6159@.microsoft.com...
>
>|||The correct way is dependant on your needs. But I would never want to give
a client direct permission to import a file with BCP to my production
server. Why not have them FTP it to a secure folder on your server or
somewhere the server can get to it. Then use Bluk Insert to load it.
Andrew J. Kelly SQL MVP
"gok" <gok@.discussions.microsoft.com> wrote in message
news:105F0D2F-92B5-427A-96FB-F17BEAABE17F@.microsoft.com...
>I have a data file (from user verification in xsl) I need to import to sql.
> the idea was to give client copy of bcp.exe and he will post data from his
> machine to sql. In BULK INSERT I have to use shared folder to import data
> from file, otherwise this file still "invisible" for sql (I was trying
> query
> analyzer on non-sql machine).
> What would be a correct way to append data from file?
> "Andrew J. Kelly" wrote:
>|||you'r right, no reason to give bcp to user. In my case it is hidden by
front-end .exe app.
Right now I dont see any advantages to use bcp (so BULK INSERT) to upsize
data: it is not secure, it cannt handle table relations and creating temp
tables on sql side has no benefits either. Better lets bring those tree-like
data directly to dbase and make sql to do all quality control checks!
"Andrew J. Kelly" wrote:

> The correct way is dependant on your needs. But I would never want to giv
e
> a client direct permission to import a file with BCP to my production
> server. Why not have them FTP it to a secure folder on your server or
> somewhere the server can get to it. Then use Bluk Insert to load it.
> --
> Andrew J. Kelly SQL MVP
>
> "gok" <gok@.discussions.microsoft.com> wrote in message
> news:105F0D2F-92B5-427A-96FB-F17BEAABE17F@.microsoft.com...
>
>