clearwing

changeset 176:f7f2a9ccb83e 1.2.3-genius

Added some knownIssues
author Stelios <stv@roe.ac.uk>
date Sat Dec 10 14:00:04 2016 +0200 (2016-12-10)
parents c01ce6c825bc
children a8c56a046083
files src/templates/knownIssues.html
line diff
     1.1 --- a/src/templates/knownIssues.html	Mon Sep 12 15:58:33 2016 +0300
     1.2 +++ b/src/templates/knownIssues.html	Sat Dec 10 14:00:04 2016 +0200
     1.3 @@ -7,109 +7,47 @@
     1.4  
     1.5  <div id="content_knownissues">
     1.6  
     1.7 -    <h3>OSA - Known Issues</h3>
     1.8  
     1.9 -    <h3>General problems:</h3><br>
    1.10 -    Please pay particular attention to the following outstanding problems:
    1.11 -    <ul>
    1.12 -    <li> 
    1.13 -    <a href="#unpaired">Unpaired sources in the atlasSource table</a>  - a small percentage of sources
    1.14 -    in atlasSource appear as separate sources in different wavebands, rather than
    1.15 -    being merged into a single source. See below for more details
    1.16 -    </li>
    1.17 +$def with (survey_prefix)
    1.18 +<script src="$survey_prefix/static/js/scrolling_helper.js" type="text/javascript" charset="utf-8"></script>
    1.19 +<script type="text/javascript">
    1.20 +                var this_div_name = 'knownissues';
    1.21 +                init_scrolling_mechanism(this_div_name);
    1.22 +</script>
    1.23  
    1.24 -    </ul>
    1.25 +<div id="content_knownissues">
    1.26  
    1.27 -    <a name="unpaired"></a><em>Unpaired sources in the atlasSource table</em>
    1.28 +    <h3>Known issues</h3>
    1.29 +
    1.30      <p>
    1.31 -    Source merging, i.e. taking the detections in various wavebands and
    1.32 -    combining them for a given source object, is performed on a frameset basis.
    1.33 -    Currently for ATLAS a frameset is based on a single detector (one of 32) 
    1.34 -    that is the result of stacking two dither offsets (to fill the gaps). </p><p>
    1.35 -    It has been discovered that some pointings in ATLAS in the various wavebands
    1.36 -    are offset from each other. As this offset increases the number of 
    1.37 -    unmerged/unpaired sources increases. In previous surveys archived by WFAU
    1.38 -    this effect has been negated by the large overlaps between pointings
    1.39 -    (eg UKIDSS) and the seamless catalogues extracted (using the priorOrSec flag)
    1.40 -    do not really suffer from the these edge effects.
    1.41 -    </p><p>
    1.42 -    Unfortunately overlap between pointings with ATLAS is small and the
    1.43 -    offsets between wavebands can be quite large compared to the framesets 
    1.44 -    themselves. The effect is compounded by the fact that spurious sources
    1.45 -    are more commonly found around the detector edges due to issues of scattered 
    1.46 -    light and single exposure coverage. An investigation based on looking at 
    1.47 -    g and r sources in DR1 showed that around 1.5 percent of real pairings are 
    1.48 -    missed. However all detections are stored in atlasDetection and every detection
    1.49 -    appears in a source in atlasSource it's just that the source merging
    1.50 -    is sub-optimnal resulting in unmatched sources in the primary selection
    1.51 -    because their match appears on a different frameset.
    1.52 -    </p><p>
    1.53 -    To visualize the issue we can look at multiframeIDs
    1.54 -    34350 (g band) and 34575 (r band). The field centres for these two 
    1.55 -    multiframes are some 40 arcsec apart. The following query finds
    1.56 -    all primary sources with g or r mags brighter than 20th.
    1.57 +      <br/>
    1.58 +      <h4>VARBINARY:</h4>
    1.59 +      Columns that are of type: VARBINARY cannot be handled by the system yet
    1.60 +      <br/><br/>  <br/>
    1.61 +      <h4>Out of date Metadata:</h4>
    1.62 +      As the system is in a prototype phase, it may not be up to date with the latest GACS TAP data release. This means that if the table or column metadata has changed, you may see e$
    1.63 +      <br/><br/>  <br/>
    1.64 +      <h4>Query issues</h4>
    1.65 +       The following types of queries can not be run at this stage:<br/><br/>
    1.66 +       <ul>
    1.67 +          <li>Filtering rows using IN <br/><i>Select .. WHERE colname IN (x,y,z)</i></li>
    1.68 +          <li>CAST()<br/><i>Select CAST(colname AS INT)</i></li>
    1.69 +          <li>Bitwise Operators<br/><i>SELECT iPetroMag AS iPetroMag ,rmiExt AS rmiExt FROM table WHERE (rppErrBits|JDBC_TABLE_507801.ippErrBits) < 65536</i></li>
    1.70 +          <li>% Symbol<br/><i>SELECT TOP 1000 ra, dec FROM table WHERE sourceID%10000 = 0</i></li>
    1.71 +          <li>SELECT table.*<br/><i>SELECT ATLASsource.*..</i></li>
    1.72 +</i></li>
    1.73 +       </ul>
    1.74 +      <h4>Memory issues</h4>
    1.75 +      <ul>
    1.76 +          <li>Certain queries that use distinct have been found to cause memory overflows in DQP processing</li>
    1.77 +         
    1.78  
    1.79 -    </p><pre>select s.ra as raDeg,s.dec as decDeg, gApermag3, rApermag3 
    1.80 -    from atlasSource as s, atlasmergelog as m
    1.81 -    where m.framesetid=s.framesetid 
    1.82 -    and (priOrSec=0 or priOrSec=s.framesetid) /* seamless primary sources*/
    1.83 -    and (gmfid=34350 or rmfid=34575)
    1.84 -    and (gApermag3 between 0 and 20  or rapermag3 between 0 and 20)
    1.85 -    </pre>
    1.86  
    1.87 -    This query reurns 3437 sources, of these 349 have either
    1.88 -    no g or no r magnitude (gapermga3 &lt;0 or rapermag3 &lt;0) 
    1.89 -    i.e. have no match in the other band. The plot below shows
    1.90 -    all the sources (blue) and unpaired (red), the unpaired mostly
    1.91 -    follow the detector/frameset edges.
    1.92 -    <p>
    1.93 -    <img src="$survey_prefix/static/images/G34350_R34575.jpg">
    1.94 -    </p><p>
    1.95 +    </p>
    1.96  
    1.97 -    We can recover matches by selecting secondary sources in the region
    1.98 -    of interest
    1.99  
   1.100 -    </p><pre>select ra as raDeg,dec as decDeg,gApermag3,rApermag3 
   1.101 -    from atlasSource
   1.102 -    where(priOrSec !=0 and priOrsec != framesetid) /* secondary */
   1.103 -    and (gApermag3 &gt;0 or rApermag3 &gt;0)
   1.104 -    and ra between 192.5 and 193.9 and
   1.105 -    dec between -7.15 and -6.05
   1.106 -    </pre>
   1.107  
   1.108 -    Pairing up the 349 unmatched sources from the first query with the results 
   1.109 -    of the second query, we find that 168 have matches. The recovered mags
   1.110 -    are plotted below showing that 163 of the matches show reasonable agreement
   1.111 -    and are likely real. The unrecovered mags will most be crud 
   1.112 -    detectiions around detector/frameset edges. So for these particular two
   1.113 -    observations around 5% of real potential g/r pairs are missing in the
   1.114 -    primary source catalogue but can be recovered.
   1.115 -    <p>
   1.116 -    <img src="$survey_prefix/static/images/G34350_R34575mags.jpg">
   1.117 -    </p><p>
   1.118 -    Contamination from unmatched sources can be reduced by including the dither 
   1.119 -    region surrounding a framesset, this is done using the 
   1.120 -    <a class="ext_links_knownissues" href="#ppErrBits_div">ppErrBit flags</a>, so
   1.121 -    the first query above becomes
   1.122  
   1.123 -    </p><pre>select s.ra as raDeg,s.dec as decDeg, gApermag3, rApermag3
   1.124 -    from atlasSource as s, atlasmergelog as m
   1.125 -    where m.framesetid=s.framesetid
   1.126 -    and (priOrSec=0 or priOrSec=s.framesetid) /* seamless primary sources*/
   1.127 -    and (gmfid=34350 or rmfid=34575)
   1.128 -    and (gApermag3 between 0 and 20  or rapermag3 between 0 and 20)
   1.129 -    and rppErrbits &amp; 0x00400000 = 0
   1.130 -    and rppErrbits &amp; 0x00400000 = 0
   1.131 -    </pre>
   1.132  
   1.133 -    This query now returns some 2828 sources, with 233 unpaired. Some
   1.134 -    129 of these are recovered by pairing with the secondary sources in this
   1.135 -    region. Although fewer unpaired sources are returned using the ppErrBit
   1.136 -    flag the downside is gaps in coverage.
   1.137 -    <p>
   1.138 -    If you have questions of the issue or need help with recovering matches
   1.139 -    please contact osa-support.
   1.140 +</div>
   1.141  
   1.142 -
   1.143 -    </p>    
   1.144 -</div>