NIBRS Frustration
The crim/data folks who read this blog will appreciate my whining ... everyone will just think I'm being typically impatient (and boring). As I was waiting for some students to stop by this afternoon, I decided to "play" with some NIBRS data (this is detailed crime incident data). I've had a couple of paper ideas for NIBRS that have been sitting on the back burner and I wanted to see if some of them would actually be feasible. That was at 3:45 p.m. It's now 5:15 p.m. and the data are still loading into SPSS. This is a victim-level file from 2002 looking only at property crimes, so not even all of the 2002 data. The case count so far is 2,360,000 and counting.
It's was fine to let this run in the background while I met with students, read my mail, cleaned up my email inbox, etc. Now I want to go home. It's after 5 p.m. on Thursday. Thursdays are my Fridays (end of the teaching week and my evening off) so I just want to go home, eat dinner, and watch The Wire. I could just leave my computer on and go home, but it's running off of the software on my PC and I'm afraid it might crash at this point or give me an incomplete file and then I won't trust the file that I have in the morning and I'll have to start all over again. I also could just start all over again tomorrow, but that seems kind of silly since it's probably at least half way done by now.
This painful lesson in patience has made me appreciate Holli Lama's complaints of working with large graphic files, and it also reminds me of why no one analyzes NIBRS data or at least not assistant professors who would like to get tenure in this lifetime!!
It's was fine to let this run in the background while I met with students, read my mail, cleaned up my email inbox, etc. Now I want to go home. It's after 5 p.m. on Thursday. Thursdays are my Fridays (end of the teaching week and my evening off) so I just want to go home, eat dinner, and watch The Wire. I could just leave my computer on and go home, but it's running off of the software on my PC and I'm afraid it might crash at this point or give me an incomplete file and then I won't trust the file that I have in the morning and I'll have to start all over again. I also could just start all over again tomorrow, but that seems kind of silly since it's probably at least half way done by now.
This painful lesson in patience has made me appreciate Holli Lama's complaints of working with large graphic files, and it also reminds me of why no one analyzes NIBRS data or at least not assistant professors who would like to get tenure in this lifetime!!
posted by Janet | 5:12 PM
3 Comments:
Two points:
(1) If you're serious about using NIBRS, you really should download all of the Segments, read them into SPSS as separate files, and then use TomZ's extract perl script to build your analysis files. It will take much less time. I downloaded the NIBRS victim file and had to use unix tools to grep out the cases of interest to me.
(2) I don't remember what I was going to say.... I suggested to Chris Maxwell that ICPSR build a server-side tool that would build custom subsets off these extracts... which is easy for me to suggest given I don't have to pay for the bandwidth or programming.
Regarding point #1, I am using Tom's extract script and this is a subset of the data ... that's why this is so frustrating. I am taking elements from multiple levels, which means that SPSS has to read in many large files to extract what I want. It's just a burden on the system and we don't have a lot of server space here so I don't think that's really an option for me either.
Regarding #2 ... Great suggestion, but I won't hold my breath.
I forgot to mention that I'm using the 2002 data because I had already downloaded all of the raw files and created SPSS system files so that I could use Tom's script. That process alone took about 5 hours!
Post a Comment
<< Home