For example, to print the double precision PI I think you are confusing internal machine precision with printįormatting precision. Does anyone know of a way to declare a variable to hold a 10 digitĪnswer: Here is Liam Gumley's reponse to this question. Me to create a 10 digit decimal number, without the number being rounded I have not been able to find a procedure allowing However I would like to create a variable corresponding to a fraction of a Precision but this seems to only hold 8 digits total for decimal numbers. Passed the decimal point, and 4 numbers before the decimal point (e.g. Question: I'd like create a varible which always holds numbers out to 6 places To the precision of the represetation (well, maybe apart from an insidious compiler bug,īut that would never happen with IDL! :o)
Idl findgen code#
There's nothing worse that trying to debug code and discovering weird results are related I reckon that, if in doubt, *always* use double precision for floating point variables. So that you'll have, say, 16-17 significant figures, or a number "good" to Have pointed out you need to create the number as a double to begin with, You'll then have a very precise, "nearly correct" number. That is, you'reĬonverting a single precision number (where the last 2 decimap points can be anything, see above) But, why doesn't x2=Double(443496.984) produce the correct result?īecause 443496.984 is, again, a single precision literal constant. Precision value to 443496.984 and print it out, a result of 443496.9XX where the XX can be So, if the 7 significant figure assumption is a good one (it may not be), then if you set a single Precision probably good to 1.100000e+37ġ.01e+37 = 3 significant figures. So, **assuming** that 7 significantįigures is a definite "cutoff" for precision, then:ġ.1e+37 = 2 significant figures. Numbers are stored with a mantissa (describes the actual number you want)Īnd an exponent (describes the, well, the exponent.). Most floating point numbers can't be represented exactly in binary (I say most because But, first, here is what Paul had to say to this person's question:ĭon't confuse the magnitude of the number with its precision. Then, I want to add a couple of other articles I think you Or seven decimal places of significance").ĪNSWER: To address this specific question, I present the excellent answer Paul van Delst offered Number this large (floats can handle "☑0^38, with approximately six
![idl findgen idl findgen](https://present5.com/presentation/566b216a53ca49e1287e3c6367875118/image-17.jpg)
Number 443496.969 from basic assignments using Float() or Double(),ĭespite the fact that even floats should easily be able to handle a The problem is that the number 443496.984 is being turned into the I ran into a number transformation error yesterday that is stillĬonfusing me this morning.
![idl findgen idl findgen](https://slidetodoc.com/presentation_image_h/fad19b53cb6307ace538f5c51de80faa/image-10.jpg)
Here is the most recent incarnation of the question: What it boils down to is that the person posting the article is just about to get his or her first See how perverse this is and blah, blah, blah.
![idl findgen idl findgen](https://satoblo.com/wp-content/uploads/2020/04/psym.jpg)
And I'm sure every thinking person with half a brain will Like this: " I've found a bug of such magnitude that it confirms my feeling that IDL is a piece of blah, blah. On the IDL newsgroup under the general category of The Sky is Falling! It usually starts out
Idl findgen how to#
QUESTION: I'm not sure exactly how to express this question.