Atlanta Custom Software Development 

 
   Search        Code/Page
 

User Login
Email

Password

 

Forgot the Password?
Services
» Web Development
» Maintenance
» Data Integration/BI
» Information Management
Programming
  Database
Automation
OS/Networking
Graphics
Links
Tools
» Regular Expr Tester
» Free Tools

How does decimal datatypes differ in their precision & scale?

Total Hit ( 1719)

Rate this article:     Poor     Excellent 

 Submit Your Question/Comment about this article

Rating


 


Click here to copy the following block
--Each numeric data type of distinct precision & scale is considered a
-- different data type. Hence result of an operation involving these will vary.
declare @one numeric(10,2) declare @zero numeric(10,2)
select @one = -1.0
select @zero = 0.0
select @one * @zero, convert(numeric(10, 2), @one * @zero)
select @one * @zero as n into #t
select * from #t
exec ('use tempdb exec sp_help ''#t''')

/* Multiplication operation & precision / scale calculation
@one (p1, s1)
@zero (p2, s2)
p1 + p2 + 1, s1 + s2
21, 4
*/


/*
If you execute the code, you will see that the CAST prints the value correctly.
I think it has got something to with the precision & scale of the resulting
data when you multiply two values. I will explain.
Consider two numeric values n1(a, x) and n2(b, y)
where a & b is the precision and x & y is the scale.
If you multiply n1*n2, then the resulting value will be n3(a + b + 1, x + y).
In your example, the value will be n3(21, 4) after the multiplication operation.
You can notice the scale from the number of zeroes after the decimal point.
This will explain why you are getting -.0000 instead of .00.
The CAST converts it to the data type numeric(10, 2) which prints .00.
To test the actual precision & scale of the converted value, add this line to the code
*/

-- SELECT @one*@zero as n INTO #t
-- Exec('use tempdb exec sp_help #t')</b><br>
/*
From the sp_help output you can see that the numeric column is indeed defined as numeric(21, 4).
Another point to note is that each numeric data type of distinct precision & scale is
considered a different data type.
Hope this info provides some insight.
*/


Submitted By : Nayan Patel  (Member Since : 5/26/2004 12:23:06 PM)

Job Description : He is the moderator of this site and currently working as an independent consultant. He works with VB.net/ASP.net, SQL Server and other MS technologies. He is MCSD.net, MCDBA and MCSE. In his free time he likes to watch funny movies and doing oil painting.
View all (893) submissions by this author  (Birth Date : 7/14/1981 )


Home   |  Comment   |  Contact Us   |  Privacy Policy   |  Terms & Conditions   |  BlogsZappySys

© 2008 BinaryWorld LLC. All rights reserved.