|
Author
|
Topic: dec min interpolation
|
Paul
|
posted February 03, 2004 10:14 AM
I know there's been a fair amount of discussion on this Pub 249 topic but in checking my scores from Quiz #5 I came to wonder about how we differed in treating this issue. The Box 5 d corr value given in the Solutions (problem #CN05-18) is 35.3, derived from dec min =38.3 where d = 56. The Pub 249 Table 5 intersection of 56/38 = 35 but so also does the intersection of 56/37 = 35. My question then is: wouldn't one assume that, at a minimum, 38.0 > 35.4? That being the case, doesn't it follow that where dec min = 38.3 the interpolated d corr would be more like 35.7?
|
|
David Burch
|
posted February 03, 2004 12:17 PM
I think there is a discussion of interpolating the d-correction on answer page A-9. It seems there should be reference to this earlier. I will come back to this point as soon as i can.--david
From: Starpath, Seattle, WA
|
|
Paul
|
posted February 03, 2004 12:58 PM
The discussion on p. A-9 (if ... the correction for 13 is 8 and the correction for 14 is 9, etc.) is precisely the reason I don't get why Problem #18 d-corr wouldn't be > the 35.3 shown in Solutions.
The other question I had, and perhaps the more important in the long run, is: How much does this (± 0.5') matter?
|
|
David Burch
|
posted February 11, 2004 11:30 PM
I apologize for the late answer, i just found your continuing question here about interpolating the d correction in Pub 249.
In problem cn05-18, the d= +56', the dec minutes are 38.3. Looking at the table section below that is cut out of the full d-correction table, you see that for d = 56',
dec min = 38, the corr = 35 and for
dec min = 39, the corr = 36
hence if we have
dec min = 38.3, the corr = 35.3
so the correction in our case is 35.3'. The interpolation will always be that simple, just use the same decimal for the correction as for the minutes. In some cases, however, the correction for neighboring values will be the same. In that case, it does not matter what the decimal is, you use the one in the table. No interpolation.
As mentioned elsewhere in the note, this interpolation is not strictly speaking justified based on the inherent precision of the tables, but we have done very many tests over the years and have always found that this helps more than it hurts.
(note that i cut this table out in the wrong place, but it is perfectly symmetric in d and ', so that will not matter)
==============
For the second part of your question, 0.5' is essentially 0.5 nmi in the final fix... at least in many cases the equivalence is pretty close. The ultimate accuracy of the celestial navigation fixes using the best procedures is about ± 0.5 nmi over all. Hence if you want to do the best you can, we cannot throw away this much accuracy.
From: Starpath, Seattle, WA
|
|
Bill Fallin
|
posted February 17, 2004 07:28 AM
Thanks David and Paul,
I also noted that the difference between the two table entries was exactly 1. This justifies using whatever the fraction was in dec mins, as (e.g. .3 X 1 = .3).
My question is: does this practice holdup if the difference is other than exactly 1? i.e. If the difference in two adjacent entries in table 5 was 2, and the decimal fraction in dec mins was .3, would we multiply 2 X .3, and use the product, .6 as an interpolation? thks, billf
|
|
Gary Rose
|
posted February 04, 2005 12:54 PM
After reading all of the above and the info on page A9 I have an idea of the rounding process that is going on. I do have to say that I was a bit confused when first looking at the answers for the problems in 5.7. I do have a question about some of the problems in 5.7 and their answers.
In problem 5.7-4 the Dec min are 41.4, the Pub 249 d is +60, the Pub 249 d.corr was interpolated to 41.4. In problem 5.7-7 the Dec min are 50.4, the Pub 249 d is -60, the Pub 249 d.corr was not interpolated to 50.4 but left at 50. Why is one interpolated to XX.4 and the other one left at XX.0, when both have the same decimal value and neither have a neighboring value the same?
I dont believe it is because of the Pub 249 d is 60, since problem 5.7-9 also has a 60 for the Pub 249 d and the Dec min is 13.4 and it was interpolated to 13.4.
|
|
David Burch
|
posted February 04, 2005 09:35 PM
i will look at this monday latest and get back to you... we can never rule out a simple error or inconsistency on our part!
From: Starpath, Seattle, WA
|
|
|
