Group 1: Modeling High-Energy Count Data Topic Ia/ Jim Chiang (JC), David van Dyk (DvD) on Esch et al EMC2 code Ia.1/ Update on status of EMC2 code (David) DvD reports it is basically working. JC wants to know, can it handle big datasets? Large swaths of the sky? DvD says it is -slow- what does JC mean about big datasets? JC emphasizes large swaths of the sky rather than necessarily huge datasets. DvD felt they could handle that. But what about test datasets? Then we wil really see. Ia.2/ Status of test data and psf Jim C can get GLAST datasets but does not want it public. Jogesh Babu (JB) says that it wil be in a pwd-protected web-site. Ia.3/ Priors for smoothing parameters? JC brings up priors again. DvD At the moment the way it is quantified is not particularly amenable to prior information. At the moment we think of them as completely artificial smoothing parameters. JC: Can we use the information from his high signal-to-noise dta to get priors for the smoothing parameters? AC: suggests "running" EMC2 on the High signal-to-noise ratio data. Then use the output of that as priors for the low-count gamma-ray data. DvD: Hmm. Problem - Our EMC algorithm, code as we have it now, does it photon-by-photon. So if you have millions of photons its going to be very very slow. We have to think about it a little bit. But maybe I'm wrong because we do bins all at the same time. OR we could replace the Poisson counts with a Gaussian model. Before we do it we could do simulations, try using really strong priors. I think Jim if you could give us that data whenever you can - if we're lucky Adam can have some pictures we can look at in a couple of weeks and then we'll have lots of questions. Topic Ib/ Image Reconstruction for RHESSI: C. Alex Young Ib.1/ The BIG question: Can we get IDL code for R-L? CAY: R-L for IDL not available, unfortunately. Ib.2/ Status of test data and psf? CAY had some disussion last week with colleague who does lots of data-analysis. There are different ways to represent the datasets. * The giant response (i.e a different PSF for each location) + data; PSFs are equal to Bessel functions; OR * data plus many PSFs Ib.3/ Can we use EMC2 for these data? DvD In principle, yes, sure. CAY: Can the algorithm handle large responses? DvD: No. In principal, yes; but not in R. CAY: What's reasonable? 64x64? DvD Yes. For now if we can do simpler examples, rather than real problems. If you have data you can post, others may be ready to try things as well. CAY: If you recall, the data are time series, wiggles in the data, described by these large response matrices. Might that be a problem for EMC2? DvD: Yes as currently written. It assumes the image space is the same shape/binning as the data-space. But its a simple change. CAY: I will continue to work on the datasets, The person he was working with is out of town. He understands R-L so he can talk well with him in this way. DvD: What we are doing, is just R-L with Bells and whistles. DvD It occurs to me, now that I am here in NC, maybe I should give an Intro. Because not everyone ehre knows what we are talking about, with EMC2. CAY: I am interested in those intro slides as well. I remember hearing David Esch talking about it, but a refresher would be good. It would be helpful for my colleague as well. DvD: OK Then I will post the slides. Group 3: Statistical Issues with Low-Count Data Topic 3b/ Poisson Data in the Presence of Background DvD: Vinay is not here so we will skip over Upper Limits for now. JB: Upper Limits and Truncation are being discussed in Tom Loredo's group as well. There will be a talk in their group in a couple of weeks. Keith Arnaud (KAA) is interested in low count rate questions. DvD: There is a paper I found, rather interesting, that I grabbed. This paper was lind of a plea for help - a physicist esentially lays out various methods people use and says why they don't meet the community needs. KAA This is a question the particle physicists wory about a whole lot. In a way its linked to VK's U-L.s CAY: Its cetainly a problem with gamma-ray astronomy. KAA Suppose you are looking at Cahndtra data. The background count is very low. Suppose you want to know what the flux is from an optical galaxy. Perhaps it has no counts. What's my measurement saying? Usually in practice I am looking at spectra, which an be cosidered a whole bunch of linked cases of this. but I thought we ought to start simple, with one pixel or bin, and ask what the statistical viewpoint is. Jogesh strongly reccomends looking at the Comments to the paper, Wasserman, van Dyk, etc as well. Some discussion of how to access these - KAA and others cannot read it. Aneta: Hello can I ask a question? One thing that was not clear to me was what they used for uncertainties in the background. DvD: As far as I recall, the background intensity, the expected background, is assumed known. Aneta: In all these examples, they always consider 3 counts as coming from the background (the expected counts). In our case it (the expected background) could be two or four. KAA: But for Chandra we know our background pretty well, could be 1/2. Aneta: But not exactly. And I still don't know,in my pixel, are there 1 or 0? DvD: A statistician would tell you to go back and measure the backgroundmore accurately. (General laughter.) DvD/Aneta: If we have maybe two or three counts, what can I say about our source? DvD Maybe KAA is right, the way to pursue it is through Vinay's problem. There are really two problems: is there a source? and, What can we say about it? Aneta speaks on how useful papers are. This 2002 one goes though a good overview. * ACTION ITEM: If we know any papers, we should copy and send papers to DvD so they can be posted. Then we will read them for homework! AC will ask Tom Loredo. Didn't he do something, a similar review to Mandelkern's paper? KAA: So my other question that I raised when I was down there was more on when tehre are the nuisance parameters. More like the X-ray spectroscopy. So you have a model that describes this spectrum, with nuisance parameters, in the low-counts regime. How do we find out what to do? DvD Yes defintely tough in the low counts regime. KAA I have a bunch of papers here to suggest. Are we overloading? DvD It will just be further down on the (readng homework) list. Send us the papers and we will look at them, later. It will be fun! DvD: Any other questions? AC: Is the R code available for us to try to use? DvD: Yes, in theory. Adam could seak to how portable it is. Maybe it would be best to wait until he has tried it on a real dataset. JB: There is an R-code for MCMC on our website. Oh, really? That sounds handy. DvD: We will take a break and be back at two, when Becca Willett and Tom Lee can be here, for the solar portion. ---------------- Group 2: Modeling Optical and Solar Images Topic 2a/ Solar Astrophysics C. Alex Young CAY: It's a big topic. So we have to decide on which pieces we want to focus on. On slide 2: A picture of the sun in a bunch of different wavelengths at once. From left to right it sort of goes higher up into the atmosphere. One of our overall goals: How can we connect these features in different wavelengths?? The underlying structures are all connected. Becca Willett (BW): Are these all of the wavelenghts? CAY: These are just a select few. All except the top are from space; one on the top right is from ground. T. Lee: All at the same time? CAY: Yes, as close as possible, BW: How many other wavelenghs? CAY: For these images, several more. And there are coronographs, showing outside the disk of the sun, as well. CAY: I've been putting together daydset to look at. Roughly 4 catagories. First one, in slide nuumber 3, is basically the magnetic field data. From basically the photospher, the visble layer of the sun. I've put together some of our questiosn: Can we calssify these sunspots? BW: Are there lost of datasets? CAY: Yes. I've puut a number on-line. Hyunsook Lee (HL): What's the scientific reason? CAY: As sunspots become more complex, their magnetic fields get more tangled and are more likley to erupt. They evolve. TLee so it starts out as an alpa, and goes to beta, and so forth. CAY: Yes. T Lee: Do they move? How are they classified? By hand? CAY: Yes. The black is negative, going into the screen, the white is positive, coming out of the screen. If you look at a complex beta-gamma-delta you see a lot of gray. TLee: Do they spin at all? CAY: Yes, they do spin. That's someting you have to correct for, The sun is rotating and spots are rotating too. CAY This next is the chromosphere. People are interested in structure. Yu see that structure in the lower left? Those aer called filaments. if you look at them from the side they look differnt. Yo are intereste4d in tracking them. Next is coronal features. This is UV light. There is a lot going on here. for example in the lower left, there is actually a wave, from an explosion, that goes of on the solar disk. We tracked it by "difference" images. These are difficult to se without doing some sort of filtering. And the one on the bottom right. There's actually material that's erupting off the disk of the sun. At the bottom is the Corona. These are faint wispy sorts of structures. But there aer the structures that, if they interact with the Earth, have a lot of effect on the atmosphere. We would like to track them automatically. WB: How much of a time series do you have? CAY: Maybe a few, maybe 10. Right now these cadences are on the order of hours. The future one, we are launching, will have a cadence of minutes. An order of magnitude more data. JB: Alex what's that black feature in the middle of the picture? CAY: That's the occulting disk. It's blocking out the bright sun, so you can see hte faint corona. CAY: So I have placed datasets from these four categories under these four directories. I don't know how people want to look at these? Also a movie of interest. Vinay's comment: Even when registered, diffent features will look slightly different at differnt wavelengths. How do we correlate them? [Alex did I get your quote right?] WB: Is there anything out there now besides people looking at them? CAY: There's a group that has put some automated ways of catching coronal CMEs. I'll make these available. There's been a little wavelet stuff, some of which I have been involved with; and some simple bandpass filters to pull out structure. As you can see one of the problems is ther's such a lot of these data. It's pretty overwhelming. Probably the two biggest issue would be: detecting and trackin CMEs. The other ould be classifying and tracking Sunspots. TLEE: (I can't follow) But he and former student (Kurt Stolling - sp?) worked on tracking of storms. And storm can merge and split. If we can ask him, we can perhaps use his method to track even when these merge and split. CAY: That would be wonderful And I'll put on the web'this place in belgium that does automatic tracking. What they do, they map it to polar coordinates. A CME gives a line. Then they fit those lines with a Hough transform. But they are fitting a curved line with a straight line. for example if its accelerating the line curves more. And their methods give only a constant velocity. So it works but you are losing physics. I'll make that available as well. CAY: Under the magnetic field one, there are two directiories. Visible, but also megnetograms. Right now, all thr files are in FITS form. I am workin to try to get them into matlab or something. I know the astronomers know about fits. But have statistic BW: I've tried, with matlab. I know theer are several FITS packages out there. And I personally have never had success with it. CAY: I have, just with the reader that comes with matlab. Should I translate them to matlab format? BW: Even if you put up just a short little matlab script that would tell me what to do. (General agreement) HL: When you say you are interested in connecting differnt images at differnt wavelengths. When you see the face of the sun and the limb of the sun, the feature appears very differently. Have people tried different methods for this? CAY: That's certainly a problem. People have tried some geometricsl transformtaions to account for the projection. There is an additional problem. Since thee are not optically thick, they are semi-transparent. So you see stuff thruogh them. Is this akin to medical images, where you're seeing structure but you're also seeing through softer tissue. I guess the answer is we're not sure how to do it. HL: So a good problem. More work to do. TLee: For tracking, for chromosphere. What are the real characteristics we want to track? CAY: You want to track how long they are, and the curvature. They have little features that poke out the side, called spines. They are affected by the magnetic field, so they tell of the underlying physics. WB: For the mgnetograms and classifying. Have you thought of fractals and complexity measures? CAY: That is one thing some colleagues of mine and I are trying, to look at a fractal f of alpha spectrum, to look at the complexity. Do you have some experience? Can we talk about that as well tomorrow? That was one of my ideas of how to classify these, with fractal dimension. TLee: It is hard to do, to measure the fractal dimension. I have tried it in my PhD days and it is hard. CAY: Yes we have had a student there working on it all summer, and they spent most of their time trying to figure out why one method seemed to work and another didn't. WB: Is this data on an ftp server or something? CAY: For security reasons we can't do an ftp server. But I could tar some of it into differnt pieces. AND sinc I am coming down, I can bring you a DvD. And you can pass it on to David. BW: That would work well. CAY/DvD Any other questions? BW: On slide 5 - are these examples - you say to classify a flare vs a filament liftoff vs a wave. Do you have examples of that? CAY: I wil bring or put up examples of that. WB: Are these from RHESSI? CAY: Actually, no; RHESSI is all low-count high energy. These are all lower energy, UV and optical, mostly.