Showing posts with label solution. Show all posts
Showing posts with label solution. Show all posts

Monday, March 19, 2012

lookup values in another table

alright, I'm sure this is a simple solution, but I really don't know much about T-SQL. I have two tables, [temp], and [SectionChanges]. I am using SQL 2005 SP2. Here are the schemas:

Code Snippet

[dbo].[temp](

[Section] [smallint] NOT NULL,

[Mnemonic] [nvarchar](50) NOT NULL,

[Test Name] [nvarchar](200) NOT NULL,

[Date] [smalldatetime] NOT NULL,

[Patient Type] [smallint] NOT NULL,

[Nurse Unit] [nvarchar](20) NULL,

[Client#] [int] NOT NULL,

[Item_Qty] [smallint] NOT NULL,

[InputDate] [smalldatetime] NOT NULL,

[Bill_Item_ID] [int] NOT NULL,

[Charge_Item_ID] [int] NOT NULL

)

[dbo].[SectionChanges](

[Mnemonic] [nvarchar](50) NOT NULL,

[Test Name] [nvarchar](200) NOT NULL,

[Section] [smallint] NOT NULL

)

The data I want to use is:

Code Snippet

SELECT Mnemonic, [Test Name], Section, Bill_Item_ID
FROM temp
WHERE (Section = 0) OR
(Section >= 18)

I want to match the two tables on Mnemonic AND [Test Name] and then update the temp table with the value of section from SectionChanges

Thank you all.

Try this:

Code Snippet

UPDATE t

SET Section = sc.Section

FROM temp t

INNER JOIN SectionChanges sc

ON t.Mnemonic = sc.Mnemonic

AND t.[Test Name] = sc.[Test Name]

WHERE (Section = 0) OR

(Section >= 18)

|||Thanks much!|||

I'm sorry, it worked once, but now it isn't working any more. I now get the error:

Column or expression 'Section' cannot be updated.

This is the exact query I was using when it worked before

Code Snippet

UPDATE t
SET Section = sc.Section
FROM temp t
INNER JOIN SectionChanges sc
ON t.Mnemonic = sc.Mnemonic
AND t.[Test Name] = sc.[Test Name]
WHERE (t.Section = 0) OR
(t.Section >= 17)

|||

Have you changed anything (ie, table definition, etc.) since the first time you ran it?

No changes to the update code?

|||

no

|||alright I got it to work again but I'm baffled. The result changes depending on the query window i enter it. In Management studio, if i right click a table and click open table, then open up the query text and replace it with the update query, it gives that error. If i right click the table and goto script table as update and then paste the query, it works fine. It also doesnt work in a SQL task in SSIS.|||

I had to change the where clause to add the alias to the Section column name (I had just copied it from your code before)

but this works:

Code Snippet

create table [dbo].[temp](

[Section] [smallint] NOT NULL,

[Mnemonic] [nvarchar](50) NOT NULL,

[Test Name] [nvarchar](200) NOT NULL,

[Date] [smalldatetime] NOT NULL,

[Patient Type] [smallint] NOT NULL,

[Nurse Unit] [nvarchar](20) NULL,

[Client#] [int] NOT NULL,

[Item_Qty] [smallint] NOT NULL,

[InputDate] [smalldatetime] NOT NULL,

[Bill_Item_ID] [int] NOT NULL,

[Charge_Item_ID] [int] NOT NULL

)

create table [dbo].[SectionChanges](

[Mnemonic] [nvarchar](50) NOT NULL,

[Test Name] [nvarchar](200) NOT NULL,

[Section] [smallint] NOT NULL

)

insert into dbo.temp values (0, 'code1', 'test1', getdate(), 0, '1', 1, 1, getdate(), 0, 0)

insert into dbo.temp values (0, 'code2', 'test2', getdate(), 0, '1', 1, 1, getdate(), 0, 0)

insert into dbo.temp values (0, 'code3', 'test3', getdate(), 0, '1', 1, 1, getdate(), 0, 0)

insert into dbo.SectionChanges values('code1', 'test1', 100)

insert into dbo.SectionChanges values('code2', 'test2', 200)

insert into dbo.SectionChanges values('code3', 'test3', 300)

UPDATE t

SET Section = sc.Section

FROM temp t

INNER JOIN SectionChanges sc

ON t.Mnemonic = sc.Mnemonic

AND t.[Test Name] = sc.[Test Name]

WHERE (sc.Section = 0) OR

(sc.Section >= 18)

select *

from temp

|||

When I right clicked on [temp] and chose "Open Table" and then replaced the SQL with the update code, it gave me the same error that you received.

I noticed though that SSMS modified the script by inserting "CROSS APPLY t" before the WHERE clause.

Removing this cross apply gets rid of the problem.

(I've never used the SQL window on an OPEN TABLE to do anything like this, I usually use "New Query"...so I have no idea why SSMS decided to change what was pasted in the window.)

As for the SSIS, can you tell me what about it isn't working? Is there an error message? What all is going on in and around the SQL Task causing the problem. Please post anything can about it.

|||

Thank you very much for your help, I found out that SSIS was just a dumb mistake on my part. It's really strange that that query window gives different results and different errors. This was 1000 times more efficent than what I was trying to do before in SSIS.

Monday, March 12, 2012

Lookup / Merge Join / Script - Howto look up values by comparing to a range of values?

Hello all,

I am trying to think my way through a solution which I believe others have probably come across... I am trying to implement a matching routine wherein I need to match an address against a high value and a low value (or, for that matter an input date vs. a start and end date) to return the desired row ... i.e. if I were to use a straight vb program I would just use the following lookup:

"SELECT DISTINCT fire_id, police_ID, fire_opt_in_out, police_opt_in_out FROM ipt_tbl " & _

" WHERE zip_code = @.zip_code AND addr_prim_lo <= @.street_number AND addr_prim_hi >= @.street_number " & _

" AND addr_prim_oe = @.addr_prim_oe AND street_pre = @.street_pre AND street_name = @.street_name " & _

" AND street_suff = @.street_suff AND street_post = @.street_post " & _

" AND (expiry_date = '' OR expiry_date = '00000000' OR expiry_date > @.expiry_date)" & _

" GROUP BY fire_ID, police_ID, fire_opt_in_out, police_opt_in_out"

My question, then, is how would you perform this type of query using a lookup / merge join or script? I have not found a way to implement a way to set the input columns? I can set the straight matches without a problem, i.e. lookup zip code = input zip code, but can't think of the correct way to set comparisons, i.e. lookup value 1 <= input value AND lookup value 2 >= input value

Any suggestions?

thanks for your time...

After a bit of research, I have found a few different solutions to this problem. The first uses the lookup by altering the sql statement and parameter set under the advanced tab (enable memory restriction / modify the sql statement) as discussed here http://www.julian-kuiters.id.au/article.php/ssis-lookup-with-range. The second (a bit less likeable) option was to use a script component and found here http://sqlblogcasts.com/blogs/simons/archive/2005/10/04/628.aspx

I am a bit disappointed that the range functionality was not prebuilt in to the components (other than through the memory restriction)... Does anyone else have any ideas or suggestions?

Thanks for your time.

|||Although the logic in your query is pretty clear, the context and output is not - at least not to me. What exactly do you want to have happen in your data flow as the output of this process?|||

Reading back I can see how you would be confused...

I am trying to look up information based on an address range (i.e. if you are on the 23rd block (2300 - 2400) there will be one set of police and fire municipality codes, if you are on the 24th block 2401 - 2500 there will be seperate municipality code outputs) We are aggregatting these facts based on addresses supplied from our customer base to a table provided by the state which maps out the various address blocks for which we will need to break down our earned premiums such that we can pay taxes to the firefighter and police pension plans.

I think that the main point was that there is not an intuitive way of looking up information based on a fact tables value being within the dimension tables value range. I have also had this problem looking up values based on date ranges...

|||

I don't know how much anyone is interested, but here is a topic on the feedback site which you can vote on to bring this to the ssis teams attention...

https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=263594

Lookup / Merge Join / Script - How to?

Hello all,

I am trying to think my way through a solution which I believe others have probably come across... I am trying to implement a matching routine wherein I need to match an address against a high value and a low value (or, for that matter an input date vs. a start and end date) to return the desired row ... i.e. if I were to use a straight vb program I would just use the following lookup:

"SELECT DISTINCT fire_id, police_ID, fire_opt_in_out, police_opt_in_out FROM ipt_tbl " & _

" WHERE zip_code = @.zip_code AND addr_prim_lo <= @.street_number AND addr_prim_hi >= @.street_number " & _

" AND addr_prim_oe = @.addr_prim_oe AND street_pre = @.street_pre AND street_name = @.street_name " & _

" AND street_suff = @.street_suff AND street_post = @.street_post " & _

" AND (expiry_date = '' OR expiry_date = '00000000' OR expiry_date > @.expiry_date)" & _

" GROUP BY fire_ID, police_ID, fire_opt_in_out, police_opt_in_out"

My question, then, is how would you perform this type of query using a lookup / merge join or script? I have not found a way to implement a way to set the input columns? I can set the straight matches without a problem, i.e. lookup zip code = input zip code, but can't think of the correct way to set comparisons, i.e. lookup value 1 <= input value AND lookup value 2 >= input value

Any suggestions?

thanks for your time...

After a bit of research, I have found a few different solutions to this problem. The first uses the lookup by altering the sql statement and parameter set under the advanced tab (enable memory restriction / modify the sql statement) as discussed here http://www.julian-kuiters.id.au/article.php/ssis-lookup-with-range. The second (a bit less likeable) option was to use a script component and found here http://sqlblogcasts.com/blogs/simons/archive/2005/10/04/628.aspx

I am a bit disappointed that the range functionality was not prebuilt in to the components (other than through the memory restriction)... Does anyone else have any ideas or suggestions?

Thanks for your time.

|||Although the logic in your query is pretty clear, the context and output is not - at least not to me. What exactly do you want to have happen in your data flow as the output of this process?|||

Reading back I can see how you would be confused...

I am trying to look up information based on an address range (i.e. if you are on the 23rd block (2300 - 2400) there will be one set of police and fire municipality codes, if you are on the 24th block 2401 - 2500 there will be seperate municipality code outputs) We are aggregatting these facts based on addresses supplied from our customer base to a table provided by the state which maps out the various address blocks for which we will need to break down our earned premiums such that we can pay taxes to the firefighter and police pension plans.

I think that the main point was that there is not an intuitive way of looking up information based on a fact tables value being within the dimension tables value range. I have also had this problem looking up values based on date ranges...

|||

I don't know how much anyone is interested, but here is a topic on the feedback site which you can vote on to bring this to the ssis teams attention...

https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=263594

Friday, March 9, 2012

Lookup / Merge Join / Script - How to?

Hello all,

I am trying to think my way through a solution which I believe others have probably come across... I am trying to implement a matching routine wherein I need to match an address against a high value and a low value (or, for that matter an input date vs. a start and end date) to return the desired row ... i.e. if I were to use a straight vb program I would just use the following lookup:

"SELECT DISTINCT fire_id, police_ID, fire_opt_in_out, police_opt_in_out FROM ipt_tbl " & _

" WHERE zip_code = @.zip_code AND addr_prim_lo <= @.street_number AND addr_prim_hi >= @.street_number " & _

" AND addr_prim_oe = @.addr_prim_oe AND street_pre = @.street_pre AND street_name = @.street_name " & _

" AND street_suff = @.street_suff AND street_post = @.street_post " & _

" AND (expiry_date = '' OR expiry_date = '00000000' OR expiry_date > @.expiry_date)" & _

" GROUP BY fire_ID, police_ID, fire_opt_in_out, police_opt_in_out"

My question, then, is how would you perform this type of query using a lookup / merge join or script? I have not found a way to implement a way to set the input columns? I can set the straight matches without a problem, i.e. lookup zip code = input zip code, but can't think of the correct way to set comparisons, i.e. lookup value 1 <= input value AND lookup value 2 >= input value

Any suggestions?

thanks for your time...

After a bit of research, I have found a few different solutions to this problem. The first uses the lookup by altering the sql statement and parameter set under the advanced tab (enable memory restriction / modify the sql statement) as discussed here http://www.julian-kuiters.id.au/article.php/ssis-lookup-with-range. The second (a bit less likeable) option was to use a script component and found here http://sqlblogcasts.com/blogs/simons/archive/2005/10/04/628.aspx

I am a bit disappointed that the range functionality was not prebuilt in to the components (other than through the memory restriction)... Does anyone else have any ideas or suggestions?

Thanks for your time.

|||Although the logic in your query is pretty clear, the context and output is not - at least not to me. What exactly do you want to have happen in your data flow as the output of this process?|||

Reading back I can see how you would be confused...

I am trying to look up information based on an address range (i.e. if you are on the 23rd block (2300 - 2400) there will be one set of police and fire municipality codes, if you are on the 24th block 2401 - 2500 there will be seperate municipality code outputs) We are aggregatting these facts based on addresses supplied from our customer base to a table provided by the state which maps out the various address blocks for which we will need to break down our earned premiums such that we can pay taxes to the firefighter and police pension plans.

I think that the main point was that there is not an intuitive way of looking up information based on a fact tables value being within the dimension tables value range. I have also had this problem looking up values based on date ranges...

|||

I don't know how much anyone is interested, but here is a topic on the feedback site which you can vote on to bring this to the ssis teams attention...

https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=263594

Saturday, February 25, 2012

Looking for Solution to Remote Data Entry

I'm looking for different solution people have implemented to solve this
problem.
Problem: People need to work remotely. While working remotely they want to
gather information. This information might be survey info, or inspection
info, etc. This information is to be stored remotely on a handheld, or
laptop device. After they get back to the office/homeo office they want to
upload the information to our corporate database, which is SQL Server 2000.
This upload process should be very easy, like a like of a button, or done
automatically when the devices is docked into the corporate network. This
solution has to be supported over the Internet, and intranet.
What I'm looking for is what technologies and processes have people put in
places to implement a solution to handle the problem I stated above.
If you have more questions about what I am asking for please let me know.Replication is the MS SQL Server technology for this.
Quentin
"Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
> I'm looking for different solution people have implemented to solve this
> problem.
> Problem: People need to work remotely. While working remotely they want
> to
> gather information. This information might be survey info, or inspection
> info, etc. This information is to be stored remotely on a handheld, or
> laptop device. After they get back to the office/homeo office they want
> to
> upload the information to our corporate database, which is SQL Server
> 2000.
> This upload process should be very easy, like a like of a button, or done
> automatically when the devices is docked into the corporate network. This
> solution has to be supported over the Internet, and intranet.
> What I'm looking for is what technologies and processes have people put in
> places to implement a solution to handle the problem I stated above.
> If you have more questions about what I am asking for please let me know.
>|||Are you saying you would use replication from a CE or Laptop device back to
your corporate database?
What other technologies might be available. I'm looking for all the
different technologies so I can determine what might be best and worst
methods.
"Quentin Ran" wrote:
> Replication is the MS SQL Server technology for this.
> Quentin
>
> "Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
> news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
> > I'm looking for different solution people have implemented to solve this
> > problem.
> >
> > Problem: People need to work remotely. While working remotely they want
> > to
> > gather information. This information might be survey info, or inspection
> > info, etc. This information is to be stored remotely on a handheld, or
> > laptop device. After they get back to the office/homeo office they want
> > to
> > upload the information to our corporate database, which is SQL Server
> > 2000.
> > This upload process should be very easy, like a like of a button, or done
> > automatically when the devices is docked into the corporate network. This
> > solution has to be supported over the Internet, and intranet.
> >
> > What I'm looking for is what technologies and processes have people put in
> > places to implement a solution to handle the problem I stated above.
> >
> > If you have more questions about what I am asking for please let me know.
> >
> >
>
>|||> Are you saying you would use replication from a CE or Laptop device back
> to
> your corporate database?
Yes that's what I am saying.
> What other technologies might be available. I'm looking for all the
> different technologies so I can determine what might be best and worst
> methods.
I am a SQL Server guy, not too much into other applications. One thing I
see can be useful is message queueing.
> "Quentin Ran" wrote:
>> Replication is the MS SQL Server technology for this.
>> Quentin
>>
>> "Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
>> news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
>> > I'm looking for different solution people have implemented to solve
>> > this
>> > problem.
>> >
>> > Problem: People need to work remotely. While working remotely they
>> > want
>> > to
>> > gather information. This information might be survey info, or
>> > inspection
>> > info, etc. This information is to be stored remotely on a handheld, or
>> > laptop device. After they get back to the office/homeo office they
>> > want
>> > to
>> > upload the information to our corporate database, which is SQL Server
>> > 2000.
>> > This upload process should be very easy, like a like of a button, or
>> > done
>> > automatically when the devices is docked into the corporate network.
>> > This
>> > solution has to be supported over the Internet, and intranet.
>> >
>> > What I'm looking for is what technologies and processes have people put
>> > in
>> > places to implement a solution to handle the problem I stated above.
>> >
>> > If you have more questions about what I am asking for please let me
>> > know.
>> >
>> >
>>

Looking for Solution to Remote Data Entry

I'm looking for different solution people have implemented to solve this
problem.
Problem: People need to work remotely. While working remotely they want to
gather information. This information might be survey info, or inspection
info, etc. This information is to be stored remotely on a handheld, or
laptop device. After they get back to the office/homeo office they want to
upload the information to our corporate database, which is SQL Server 2000.
This upload process should be very easy, like a like of a button, or done
automatically when the devices is docked into the corporate network. This
solution has to be supported over the Internet, and intranet.
What I'm looking for is what technologies and processes have people put in
places to implement a solution to handle the problem I stated above.
If you have more questions about what I am asking for please let me know.
Replication is the MS SQL Server technology for this.
Quentin
"Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
> I'm looking for different solution people have implemented to solve this
> problem.
> Problem: People need to work remotely. While working remotely they want
> to
> gather information. This information might be survey info, or inspection
> info, etc. This information is to be stored remotely on a handheld, or
> laptop device. After they get back to the office/homeo office they want
> to
> upload the information to our corporate database, which is SQL Server
> 2000.
> This upload process should be very easy, like a like of a button, or done
> automatically when the devices is docked into the corporate network. This
> solution has to be supported over the Internet, and intranet.
> What I'm looking for is what technologies and processes have people put in
> places to implement a solution to handle the problem I stated above.
> If you have more questions about what I am asking for please let me know.
>
|||Are you saying you would use replication from a CE or Laptop device back to
your corporate database?
What other technologies might be available. I'm looking for all the
different technologies so I can determine what might be best and worst
methods.
"Quentin Ran" wrote:

> Replication is the MS SQL Server technology for this.
> Quentin
>
> "Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
> news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
>
>
|||> Are you saying you would use replication from a CE or Laptop device back
> to
> your corporate database?
Yes that's what I am saying.

> What other technologies might be available. I'm looking for all the
> different technologies so I can determine what might be best and worst
> methods.
I am a SQL Server guy, not too much into other applications. One thing I
see can be useful is message queueing.
[vbcol=seagreen]
> "Quentin Ran" wrote:

Looking for Solution to Remote Data Entry

I'm looking for different solution people have implemented to solve this
problem.
Problem: People need to work remotely. While working remotely they want to
gather information. This information might be survey info, or inspection
info, etc. This information is to be stored remotely on a handheld, or
laptop device. After they get back to the office/homeo office they want to
upload the information to our corporate database, which is SQL Server 2000.
This upload process should be very easy, like a like of a button, or done
automatically when the devices is docked into the corporate network. This
solution has to be supported over the Internet, and intranet.
What I'm looking for is what technologies and processes have people put in
places to implement a solution to handle the problem I stated above.
If you have more questions about what I am asking for please let me know.Replication is the MS SQL Server technology for this.
Quentin
"Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
> I'm looking for different solution people have implemented to solve this
> problem.
> Problem: People need to work remotely. While working remotely they want
> to
> gather information. This information might be survey info, or inspection
> info, etc. This information is to be stored remotely on a handheld, or
> laptop device. After they get back to the office/homeo office they want
> to
> upload the information to our corporate database, which is SQL Server
> 2000.
> This upload process should be very easy, like a like of a button, or done
> automatically when the devices is docked into the corporate network. This
> solution has to be supported over the Internet, and intranet.
> What I'm looking for is what technologies and processes have people put in
> places to implement a solution to handle the problem I stated above.
> If you have more questions about what I am asking for please let me know.
>|||Are you saying you would use replication from a CE or Laptop device back to
your corporate database?
What other technologies might be available. I'm looking for all the
different technologies so I can determine what might be best and worst
methods.
"Quentin Ran" wrote:

> Replication is the MS SQL Server technology for this.
> Quentin
>
> "Greg Larsen" <GregLarsen@.discussions.microsoft.com> wrote in message
> news:C353A7E9-0D0E-4DA4-85E8-6EE8F6D754AD@.microsoft.com...
>
>|||> Are you saying you would use replication from a CE or Laptop device back
> to
> your corporate database?
Yes that's what I am saying.

> What other technologies might be available. I'm looking for all the
> different technologies so I can determine what might be best and worst
> methods.
I am a SQL Server guy, not too much into other applications. One thing I
see can be useful is message queueing.
[vbcol=seagreen]
> "Quentin Ran" wrote:
>

Looking for options to fix a poorly implemented solution

So, I've started a new job recently where I am doing work on a SQL
Server database designed by a third party software company, which also
wrote the client software accompanying the database. The client
software is crap, the data model sucks rocks, and I'm stuck with it.
So in no way can I modify client code, or redesign any element of
relational model. I've been able to modify stored procedures and
triggers for performance, perform regular maintenance tasks, and
upgrade all upgradable elements, but that's about it. On top of that,
people developed an internal method of using the datbase that is
causing it to grow way too fast. Basically, there is one table with a
BLOB field that people are using to store Word documents containing
scanned images (part of their efforts to go to a paperless system).
Documents are scanned only when there is no other way to get the
information into the system (client signatures, legal documents from
courts, etc). And I've been charged with getting the size of the
database down. Yay.
Brainstorming for solutions, I was wondering if there is any possible
way to compress BLOB fields in a way that is completely transparent to
the client. I can't think of a way, so I have little hope for that
idea. Another is to go in and make sure all of the images are
compressed. That will only yield marginal results. Finally, going in
and replacing all of the scanned documents with documents that point
to a UNC path with all of the extracted documents is a solution that
will required quite a bit of work, but is possible. (If it were up to
me, I'd just say to heck with it and keep some documents in a paper
system). Any thought about what I could do?
Thanks!When I was at Sprint, the Place Where Consultants Go To Be Punished, our
department got into a document archiving frenzy. Ended up storing all
documents into PDFs, storing them on the server, with only a link to the
document in the database.
- Wm
"AAAWalrus" <aaawalrus@.yahoo.com> wrote in message
news:8b266bc2.0312101204.32a99d74@.posting.google.com...
> So, I've started a new job recently where I am doing work on a SQL
> Server database designed by a third party software company, which also
> wrote the client software accompanying the database. The client
> software is crap, the data model sucks rocks, and I'm stuck with it.
> So in no way can I modify client code, or redesign any element of
> relational model. I've been able to modify stored procedures and
> triggers for performance, perform regular maintenance tasks, and
> upgrade all upgradable elements, but that's about it. On top of that,
> people developed an internal method of using the datbase that is
> causing it to grow way too fast. Basically, there is one table with a
> BLOB field that people are using to store Word documents containing
> scanned images (part of their efforts to go to a paperless system).
> Documents are scanned only when there is no other way to get the
> information into the system (client signatures, legal documents from
> courts, etc). And I've been charged with getting the size of the
> database down. Yay.
> Brainstorming for solutions, I was wondering if there is any possible
> way to compress BLOB fields in a way that is completely transparent to
> the client. I can't think of a way, so I have little hope for that
> idea. Another is to go in and make sure all of the images are
> compressed. That will only yield marginal results. Finally, going in
> and replacing all of the scanned documents with documents that point
> to a UNC path with all of the extracted documents is a solution that
> will required quite a bit of work, but is possible. (If it were up to
> me, I'd just say to heck with it and keep some documents in a paper
> system). Any thought about what I could do?
> Thanks!

Monday, February 20, 2012

Looking for HA solution that eliminates data loss

What High Availability/Disaster Recovery Solutions are out there to prevent
any data loss when a server or site goes down.
I know there is clustering services. Are there any other ways of achieving
no data loss. I can think of 2 phase commit , but that introduces latency I
would imagine.
Also is there a way to handle 2 phase commit outside the application level.
I believe the 2 phase commit has to be programmed with SQL right ?
Looking to hear the solutions you use to prevent from a server and site
outage
See if this helps:
http://www.microsoft.com/sql/techinf...ilability.mspx
David Portas
SQL Server MVP
|||In addition to David's post, SQL Server 2005 will have database mirroring, which is no-loss
mirroring of data with failover in only a few seconds. More information on SM SQL Server home page.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"Hassan" <fatima_ja@.hotmail.com> wrote in message news:e%23faQBLmFHA.708@.TK2MSFTNGP09.phx.gbl...
> What High Availability/Disaster Recovery Solutions are out there to prevent
> any data loss when a server or site goes down.
> I know there is clustering services. Are there any other ways of achieving
> no data loss. I can think of 2 phase commit , but that introduces latency I
> would imagine.
> Also is there a way to handle 2 phase commit outside the application level.
> I believe the 2 phase commit has to be programmed with SQL right ?
> Looking to hear the solutions you use to prevent from a server and site
> outage
>
>

Looking for HA solution that eliminates data loss

What High Availability/Disaster Recovery Solutions are out there to prevent
any data loss when a server or site goes down.
I know there is clustering services. Are there any other ways of achieving
no data loss. I can think of 2 phase commit , but that introduces latency I
would imagine.
Also is there a way to handle 2 phase commit outside the application level.
I believe the 2 phase commit has to be programmed with SQL right ?
Looking to hear the solutions you use to prevent from a server and site
outageSee if this helps:
http://www.microsoft.com/sql/techinfo/administration/2000/availability.mspx
--
David Portas
SQL Server MVP
--|||In addition to David's post, SQL Server 2005 will have database mirroring, which is no-loss
mirroring of data with failover in only a few seconds. More information on SM SQL Server home page.
--
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"Hassan" <fatima_ja@.hotmail.com> wrote in message news:e%23faQBLmFHA.708@.TK2MSFTNGP09.phx.gbl...
> What High Availability/Disaster Recovery Solutions are out there to prevent
> any data loss when a server or site goes down.
> I know there is clustering services. Are there any other ways of achieving
> no data loss. I can think of 2 phase commit , but that introduces latency I
> would imagine.
> Also is there a way to handle 2 phase commit outside the application level.
> I believe the 2 phase commit has to be programmed with SQL right ?
> Looking to hear the solutions you use to prevent from a server and site
> outage
>
>

Looking for HA solution that eliminates data loss

What High Availability/Disaster Recovery Solutions are out there to prevent
any data loss when a server or site goes down.
I know there is clustering services. Are there any other ways of achieving
no data loss. I can think of 2 phase commit , but that introduces latency I
would imagine.
Also is there a way to handle 2 phase commit outside the application level.
I believe the 2 phase commit has to be programmed with SQL right ?
Looking to hear the solutions you use to prevent from a server and site
outageSee if this helps:
http://www.microsoft.com/sql/techin...ailability.mspx
David Portas
SQL Server MVP
--|||In addition to David's post, SQL Server 2005 will have database mirroring, w
hich is no-loss
mirroring of data with failover in only a few seconds. More information on S
M SQL Server home page.
Tibor Karaszi, SQL Server MVP
http://www.karaszi.com/sqlserver/default.asp
http://www.solidqualitylearning.com/
Blog: http://solidqualitylearning.com/blogs/tibor/
"Hassan" <fatima_ja@.hotmail.com> wrote in message news:e%23faQBLmFHA.708@.TK2MSFTNGP09.phx.gb
l...
> What High Availability/Disaster Recovery Solutions are out there to preven
t
> any data loss when a server or site goes down.
> I know there is clustering services. Are there any other ways of achieving
> no data loss. I can think of 2 phase commit , but that introduces latency
I
> would imagine.
> Also is there a way to handle 2 phase commit outside the application level
.
> I believe the 2 phase commit has to be programmed with SQL right ?
> Looking to hear the solutions you use to prevent from a server and site
> outage
>
>

looking for equivalent to alphanumeric identity column

hello everyone,
i'm looking for a solution that would generate the equivalent of a
case-inspecific, alphanumeric identity column.
which is to say a column that iterates through guaranteed unique
values, but using case-inspecific alphanumeric characters instead of
just an integer value. or, another way to think of it would be a base
36 number, perhaps.
i have written a method that will turn turn a 32 bit integer value into
the appropriate 6-character string, shaving off some of the higher
order bits which are excess. so i was thinking i could just keep a
table that stores the "last used integer" and when a new alphanumeric
identity is needed, get the last used integer value, increment it,
convert it to a string for use as the alphanumeric, then update the
table with the new integer value.
would that be transactionally sound, though? for example, could someone
come in after another person read the value, but before the the table
was updated with the incremented value, and end up with duplicate
alphanumerics?
any other solutions also welcome.
NOTE: one possibly complicating factor is that while most of the
alphanumeric strings can just be the alphanumeric version of the
integer, one of the strings actually has to be a bit-scrambled version
of the integer. i've already written the routine to do the
bit-scrambling as well, but that's something to consider when making
suggestions. if there's a better way to a random or semi-random looking
sequence of strings out of what is originally an incrementing integer
value, feel free to suggest that as well!
thanks for any help,
jason>> if there's a better way to a random or semi-random looking sequence of st
rings out of what is originally an incrementing integer value, feel free to
suggest that as well! <<
Here is an implementation of the additive congruential method of
generating values in pseudo-random order and is due to Roy Hann of
Rational Commerce Limited, a CA-Ingres consulting firm. It is based on
a shift-register and an XOR-gate, and it has its origins in
cryptography. While there are other ways to do this, this code is nice
because:
1) The algorithm can be written in C or another low level language for
speed. But math is fairly simple even in base ten.
2) The algorithm tends to generate successive values that are (usually)
"far apart", which is handy for improving the performance of tree
indexes. You will tend to put data on separate physical data pages in
storage.
3) The algorithm does not cycle until it has generated every possible
value, so we don't have to worry about duplicates. Just count how many
calls have been made to the generator.
4) The algorithm produces uniformly distributed values, which is a nice
mathematical property to have. It also does not include zero.
Generalizing the algorithm to arbitrary binary word sizes, and
therefore longer number sequences, is not as easy as you might think.
Finding the "tap" positions where bits are extracted for feedback
varies according to the word-size in an extremely non-obvious way.
Choosing incorrect tap positions results in an incomplete and usually
very short cycle, which is unusable. If you want the details and tap
positions for words of one to 100 bits, see E. J. Watson, "Primitive
Polynomials (Mod 2)", Mathematics of Computation, v.16, 1962,
p.368-369. Here is code for a 31-bit integer, which you can use:
UPDATE Generator31
SET keyval =
keyval/2 + MOD(MOD(keyval, 2) + MOD(keyval/8, 2), 2)*2^30;
Or if you prefer, the algorithm in C:
int Generator31 ()
{static int n = 1;
n = n >> 1 | ((n^n >> 3) & 1) << 30;
return n;
}|||wow, this looks perfect. much better than the NOT and bit array
shuffling approach i have been tinkering with so far. thanks so much
for all the information!
a question about the transactional timing of the SQL version of this
algorithm: what is the most reliable way to get the value of keyval in
such a way as to ensure that the procedure calling the update statement
is the only procedure that receives the corresponding keyval value.
sort of like a classic semaphore, read-and-set? will putting the update
and select statements into a transaction accomplish this singularity of
selected keyval values per update?
thanks again,
jason|||hmm, having trouble finding the E. J. Watson article you mention at my
local college. the only reference i've found to it is in a mathsci
journal database online, which costs a few thousand per year to
subscribe to :)
i assume this article would give me the key "tap" points with which to
modify the algorithm to apply to bit patterns of different size (such
as 32 instead of 31)?|||1) Do updates and the audit stuff with a SERIALIZABLE isolation level
for the procedure. And look at tabel lockign options.
2) Look up "Roy Hann" and "Ingres" for an article on this. You do not
want to do 32 bits -- it is messy and causes an overflaow on a 32 bit
machine. We are talking about over 2 billion numbers already; are you
McDonald's?|||thanks for the feedback.
re: 2) hehe, well, i was hoping for a 6 character alphanumeric string.
that's inherently 36^6 possibilities, which is just a tiny bit more
than what can fit in a 31 bit pattern. though i suppose i wouldn't have
to drop a whole character from the string. i could simply stick to
whatever 6 string values are available in the 31 bit pattern.
ideally, i could find a way to incorporate the restriction i'm imposing
on the possible string values, which is no more than 2 consecutive
characters (to avoid accidental curse words and such). that
significantly reduces the number of combinations to well within the 31
bit range, but i'm not sure how i could collapse that reduced range
into the bit pattern. it seems like i could only iterate through the
bit patterns with this algorithm, test if it is a "valid" bit pattern
with regard to the character restriction, and if so, use it, if not,
iterate again.
to to somehow iterate through the bit pattern, and have every number
correspond to a known valid string, would involve coming up with some
kind of lookup-table where the valid values have been collapsed, and
are available by some kind of iterable index? that seems like a lot of
work, and all it would do is: (1) let me reduce the size of the bit
pattern from 31 to 30 bits, and (2) allow me to get ALL of the possible
values in the range, instead of those within the 31 bit pattern range.
i doubt it's worth it.|||just found Roy Hann's article called "Key Points about Surrogate Keys"
... holy crap, i love this man. this article is awesome. thanks so much
for suggesting it, it is EXACTLY what i need. covers the algorithmic
and database performance concerns all at once!
jason|||


Create function dbo.IncrementingAlphaNumericKey (@.priorAphaNumKey varchar(10))



Returns varchar(10)



/*



select dbo.IncrementingAlphaNumericKey ('BT00000ZZZ')



*/



BEGIN --function



declare @.singleChar char(1),



@.fieldLength int,



@.nextAphaNumKey varchar(10)



set @.fieldLength = Len(@.priorAphaNumKey)



-- select priorAphaNumKey = @.priorAphaNumKey,



-- fieldLength = Cast(@.fieldLength as varchar(4)),



-- lastChar = substring(@.priorAphaNumKey, @.fieldLength, 1),



-- stuff(@.priorAphaNumKey, @.fieldLength, 1,'A')



 



While @.fieldLength > 0



Begin



set @.singleChar = Substring(@.priorAphaNumKey, @.fieldLength, 1)



If @.singleChar = 'Z'



Begin



set @.nextAphaNumKey = Stuff(@.priorAphaNumKey, @.fieldLength, 1,'A')



--Print 'PriorAphaNumKey = ' + @.priorAphaNumKey + ', nextAphaNumKey = ' + @.nextAphaNumKey



End



Else



Begin



set @.singleChar = Char(ASCII(@.singleChar) + 1)



set @.nextAphaNumKey = Stuff(@.priorAphaNumKey, @.fieldLength, 1,@.singleChar)



--Print 'priorAphaNumKey = ' + @.priorAphaNumKey + ' , nextAphaNumKey = ' + @.nextAphaNumKey



break



End



set @.fieldLength = @.fieldLength - 1



set @.priorAphaNumKey = @.nextAphaNumKey --returning to the



End --While loop



 -- Print @.nextAphaNumKey



Return @.nextAphaNumKey



END --function