The need to publish scientific datasets and code

One of the interesting angles of the so-called CLIMATEGATE scandal, where files and e-mails of climate change researchers were stolen and selectively published, is the READ_ME file documenting someone's work trying to read old data files and getting models to work. Whatever ethical problems the e-mails reveal -- and I agree there are some -- the file is solid documentation that these are real scientists grappling with problems that are all too familiar to me as an astronomer, especially because I have programmed in IDL and Fortran. It seems physicists and software developers have the same reaction. Here's a sample from the file:

4. Successfully ran the IDL regridding routine quick_interp_tdm.pro (why IDL?! Why not F90?!) to produce '.glo' files.

5. Currently trying to convert .glo files to .grim files so that we can compare with previous output. However the progam suite headed by globulk.f90 is not playing nicely - problems with it expecting a defunct file system (all path widths were 80ch, have been globally changed to 160ch) and also no guidance on which reference files to choose. It also doesn't seem to like files being in any directory other than the current one!!

6. Temporarily abandoned 5., getting closer but there's always another problem to be evaded. Instead, will try using rawtogrim.f90 to convert straight to GRIM. This will include non-land cells but for comparison purposes that shouldn't be a big problem... [edit] noo, that's not gonna work either, it asks for a 'template grim filepath', no idea what it wants (as usual) and a serach for files with 'grim' or 'template' in them does not bear useful fruit. As per usual. Giving up on this approach altogether.

I have any number of text files that look like this, though I doubt as I am quite as good in documenting each step anymore, since I don't work in big projects lately. One of the worst aspects is trying to figure out what someone else really did -- whether it's regridding temperature data stored in binary files (as in the stolen file), or seeing how the 2MASS pipeline really calculates PSF and aperture photmetry, and where the aperture corrections come from (as in my life as a Staff Scientist at IPAC.) More generally, NASA and all astronomers historically had a lot of problems preserving old data since file formats were not standardized decades ago. The days when the data are a photographic plate that can be preserved in a vault for a century are long gone. In short, the READ_ME file is a snapshot of what many scientists' work is like.

Also interesting is what this reveals about how scientists should publish today. My friend David Hogg is a great advocate for publishing the code along with data and model results in journal articles. I think he's correct and astronomical (and climate!) journals should work to encourage this is in the future. I'm on the Spitzer Science Users Panel, and in a recent meeting we recommended that the Spitzer Science Center release all the computer code used to process Spitzer Space Telescope data. All this pipeline will not compile or work on other computers, it does provide a precise documentation of what was really done, and if some poor future grad student finds himself recreating it, at least he or she has a better chance of getting the answer right in less time. Furthermore, it may be useful as a model to other projects. Given the public controversy over global warming, publishing the full model codes and datasets needs to be encouraged. It may be reveal some sloppy comments or evenmistakes, but in the long run it will benefit the scientific community and the broader public.

History of Science

The Royal Society has published online a number of historic scientific documents. I particularly like the letter from Ben Franklin on the famous kite experiment:

The kite is to raised, when a thunder-gust appears to be coming on, (which is very frequent in this country)...

NASA Internships

I attended a very interesting talk today at the annual Delaware Space Grant Consortium meeting by Margaret Maher of NASA's Goddard Space Flight Center. She described their active internship program. I came away with the impression that there are many internships and they are a key path to permanent positions. I understand NASA has the goal of hiring recent graduates. Our physics and astronomy students should definitely keep these opportunities in mind. Some Useful Links:

NASA Jobs Website.
Goddard Internships.
NASA Undergraduate Internships

AstroBetter

The last time I learned a really useful trick on the Mac it was from visiting Kelle Cruz.Now I can just read her blog, AstroBetter: Tips and Tricks for Professional Astronomers. Thanks Kelle!

A Night of Astronomy at the White House (video)

President Obama Presents a Night of Astronomy from White House on Vimeo.

Also at youtube. You know, it's funny to hear him say you may discover a star, because it really is real for me. There's a picture of a "star" (or at least a "brown dwarf") I discovered in the textbook I'm using in class. In fact, it has a wikipedia page.

Astronomy at the White House

First Jazz at the White House, now Astronomy at the White House:

President Obama, having spent much of the day pondering Afghanistan, spent a few seconds Wednesday night looking through a telescope at a double-star system roughly one quadrillion miles away... It was 400 years ago, the president told the students, that Galileo built his first telescope and began probing the universe. "Galileo changed the world when he pointed his telescope to the sky. Now it's your turn," Obama said. "Don't let anyone tell ya that there isn't more to discover.

No hints though on NASA funding

Non-Academic Astronomers Network

My friend Rolf Danner has started a Facebook page for the Non-Academic Astronomers Network.

This network was originally founded as an activity of the Committee on Employment of the American Astronomical Society. The purpose of the group is to connect astronomers on a variety of career paths. If you are an astronomer working in a non-traditional job consider becoming a member.

Physics Bachelors Degree Salary Information

The AIP has put out a new flyer (PDF) with starting salaries for new college graduates. It compares offers for Physics ($40K-$63K) majors to other fields, and they do pretty well.

As we are all painfully aware, the US economy has been in a serious recession. However, we are happy to report that recent physics bachelors have been getting among the highest starting salaries. Physics bachelor’s salaries are comparable to those offered to computer science bachelor’s and graduates in many engineering fields. Physicists’ starting salaries are quite a bit higher than people who earned bachelor’s degrees in other sciences, business, the humanities, or education. These data were collected and published by the National Association of Colleges and Employers. The data reflect salary offers made by campus recruiters during the last academic year. You can find the data on the SRC website at: http://www.aip.org/statistics/trends/phystrends.html

The Gig

While I continue to procrastinate on writing a real post, I'll just point you to another new blog, The Gig by Nate Chinen. Nate is a great jazz writer.

Twitter

I joined twitter. I have no intention of using it, but who knows? My username is johngizis.

First Post

This is a test. I intend to start a personal blog.