Home

Forums

Web development

 

 

 

 
     
 
dna88 Web development and Technology Forum
 
Profile   Register   Memberlist   Usergroups   FAQ   Search  Log in
floating point variable matching problem

 
Post new topic   Reply to topic    dna88 Forum Index -> Databases Discussion Forum
Author Message
tanveer
User
User


Joined: 21 Jun 2004
Posts: 85
Location: Dhaka,Bangladesh

Post Post subject: floating point variable matching problem Reply with quote

Hi friends,
I am building a database in MySQL and there is a field cgpa which I declared as float(5,3) now the problem is when i write a query like this :-
select * from graduate where cgpa=3.456;
It returns the output Empty set.
But it works if I write select * from graduate where cgpa>3.456;

After a little research I found that If i declare it as cgpa decimal (5,3) not null then the above query runs just fine.
hanging out for ur best suggestions.
Tue Jun 29, 04 10:06 am
Back to top
tanveer View user's profile Send private message
dude
Power User
Power User


Joined: 10 Mar 2004
Posts: 376
Location: Savar, Dhaka

Post Post subject: Comparing floating point variable in MYSQL Reply with quote

Simply stated, if you are comparing FLOAT columns with numbers that have decimals, you can't use '='. This problem is common in most computer languages because floating-point values are not exact values. In most cases, changing the FLOAT to a DOUBLE will fix this.

Floating-point numbers cause confusion sometimes, because these numbers are not stored as exact values inside computer architecture. What one can see on the screen usually is not the exact value of the number.

Field types FLOAT, DOUBLE and DECIMAL are such.

CREATE TABLE t1 (i INT, d1 DECIMAL(9,2), d2 DECIMAL(9,2));
INSERT INTO t1 VALUES (1, 101.40, 21.40), (1, -80.00, 0.00),
(2, 0.00, 0.00), (2, -13.20, 0.00), (2, 59.60, 46.40),
(2, 30.40, 30.40), (3, 37.00, 7.40), (3, -29.60, 0.00),
(4, 60.00, 15.40), (4, -10.60, 0.00), (4, -34.00, 0.00),
(5, 33.00, 0.00), (5, -25.80, 0.00), (5, 0.00, 7.20),
(6, 0.00, 0.00), (6, -51.40, 0.00);

mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b
-> FROM t1 GROUP BY i HAVING a <> b;
+------+--------+-------+
| i | a | b |
+------+--------+-------+
| 1 | 21.40 | 21.40 |
| 2 | 76.80 | 76.80 |
| 3 | 7.40 | 7.40 |
| 4 | 15.40 | 15.40 |
| 5 | 7.20 | 7.20 |
| 6 | -51.40 | 0.00 |
+------+--------+-------+

The result is correct. Although the first five records look like they shouldn't pass the comparison test, they may do so because the difference between the numbers show up around tenth decimal, or so depending on computer architecture.

The problem cannot be solved by using ROUND() (or similar function), because the result is still a floating-point number. Example:

mysql> SELECT i, ROUND(SUM(d1), 2) AS a, ROUND(SUM(d2), 2) AS b
-> FROM t1 GROUP BY i HAVING a <> b;
+------+--------+-------+
| i | a | b |
+------+--------+-------+
| 1 | 21.40 | 21.40 |
| 2 | 76.80 | 76.80 |
| 3 | 7.40 | 7.40 |
| 4 | 15.40 | 15.40 |
| 5 | 7.20 | 7.20 |
| 6 | -51.40 | 0.00 |
+------+--------+-------+

This is what the numbers in column 'a' look like:

mysql> SELECT i, ROUND(SUM(d1), 2)*1.0000000000000000 AS a,
-> ROUND(SUM(d2), 2) AS b FROM t1 GROUP BY i HAVING a <> b;
+------+----------------------+-------+
| i | a | b |
+------+----------------------+-------+
| 1 | 21.3999999999999986 | 21.40 |
| 2 | 76.7999999999999972 | 76.80 |
| 3 | 7.4000000000000004 | 7.40 |
| 4 | 15.4000000000000004 | 15.40 |
| 5 | 7.2000000000000002 | 7.20 |
| 6 | -51.3999999999999986 | 0.00 |
+------+----------------------+-------+

Depending on the computer architecture you may or may not see similar results. Each CPU may evaluate floating-point numbers differently. For example in some machines you may get 'right' results by multiplying both arguments with 1, an example follows.

WARNING: NEVER TRUST THIS METHOD IN YOUR APPLICATION, THIS IS AN EXAMPLE OF A WRONG METHOD!!!

mysql> SELECT i, ROUND(SUM(d1), 2)*1 AS a, ROUND(SUM(d2), 2)*1 AS b
-> FROM t1 GROUP BY i HAVING a <> b;
+------+--------+------+
| i | a | b |
+------+--------+------+
| 6 | -51.40 | 0.00 |
+------+--------+------+

The reason why the above example seems to be working is that on the particular machine where the test was done, the CPU floating-point arithmetics happens to round the numbers to same, but there is no rule that any CPU should do so, so it cannot be trusted.

The correct way to do floating-point number comparison is to first decide on what is the wanted tolerance between the numbers and then do the comparison against the tolerance number. For example, if we agree on that floating-point numbers should be regarded the same, if they are same with precision of one of ten thousand (0.0001), the comparison should be done like this:

mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b FROM t1
-> GROUP BY i HAVING ABS(a - b) > 0.0001;
+------+--------+------+
| i | a | b |
+------+--------+------+
| 6 | -51.40 | 0.00 |
+------+--------+------+
1 row in set (0.00 sec)

And vice versa, if we wanted to get rows where the numbers are the same, the test would be:

mysql> SELECT i, SUM(d1) AS a, SUM(d2) AS b FROM t1
-> GROUP BY i HAVING ABS(a - b) < 0.0001;
+------+-------+-------+
| i | a | b |
+------+-------+-------+
| 1 | 21.40 | 21.40 |
| 2 | 76.80 | 76.80 |
| 3 | 7.40 | 7.40 |
| 4 | 15.40 | 15.40 |
| 5 | 7.20 | 7.20 |
+------+-------+-------+

The decimal works for you because DECIMAL[(M[,D])] [UNSIGNED] [ZEROFILL] is an unpacked floating-point number. Behaves like a CHAR column: ``unpacked'' means the number is stored as a string, using one character for each digit of the value. So in this case you are actually comparing two strings and '=' signs will work just fine.

I think, for such simple job go with decimal, if there is no other specific reason to use float. Saves you space and processing power. Not to mention the confusions.
Wed Jun 30, 04 2:03 am
Back to top
dude View user's profile Send private message Visit poster's website
tanveer
User
User


Joined: 21 Jun 2004
Posts: 85
Location: Dhaka,Bangladesh

Post Post subject: Re: Comparing floating point variable in MYSQL Reply with quote

Thanz very much for ur reply.
But the problem is I am doing a project in which I am buiding the Alumni section of a university dynamic+online. So for student verfication I used CGPA and at the time of database design I declared it as float. Under whom I am developing this wants me to declare it as float bcz previously a group of students built their result processing system where they declare CGPA as float. Now when they will merge both RPS and Alumni then there should not any conflict on data type.
Is there any way I can do the comparison having the CGPA field as float?
Wed Jun 30, 04 12:44 pm
Back to top
tanveer View user's profile Send private message
dinangkur
Super Moderator
Super Moderator


Joined: 24 Mar 2004
Posts: 491
Location: Dhaka, Bangladesh

Post Post subject: Reply with quote

Mr. Tavir,

As you see, this is a classic problem with MySQL. And in 2000 they didn't had float datatype. They promote to use decimal datatype. Now, according to your problem, try with FORMAT() and see if it works. If not then try with CAST() and CONVERT(). I think it will work. I will check it by myself and let you know :( too busy with exam. Hope to find a solution, after my exam.

-DK.
_________________
...we too are stardust...
Wed Jun 30, 04 1:37 pm
Back to top
dinangkur View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger
dude
Power User
Power User


Joined: 10 Mar 2004
Posts: 376
Location: Savar, Dhaka

Post Post subject: Reply with quote

Nothing else comes to my mind except for what DK suggests about trying to convert the float value before you do the comparasion. One round about thing you can do is I think, get all the values of the resultset, convert them to integer and then match the pattern and display the results. This would be inefficient but will do the job.
Fri Jul 02, 04 1:13 am
Back to top
dude View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    dna88 Forum Index -> Databases Discussion Forum All times are GMT - 7 Hours
Page 1 of 1

 

Partners and Resources

Bangladesh hosting company

Bangladesh web design

Driven by phpBB © phpBB Group