IDA@Lehman

Summary of the alumni survey

We carried out a mail survey of sociology graduates from 2001 through 2007 in order to get a sense of whether there are systematic changes. We received 111 responses from 500 surveys sent out.

It is impossible to make strong causal claims about the impact of the IDA program on sociology alumni. Factors which should lead us to interpret the results with caution are:

1. Other curricular and enrollment changes were happening at Lehman College, the City University of New York and New York City high schools during the same time period.
2. The group receiving the intervention (including those graduating in June 2007) has had less time in the post college workforce or education programs on which to base their self-assessments.
3. It is difficult to assess dosage accurately.
4. There are demographic differences between the two groups. The IDA group is more likely to have been born in another country (and less likely to have been born in Puerto Rico), to report English as a first language, and to have resided outside of the Bronx during college. (see Table 1).
5. Alumni in the IDA group self-report higher average GPAs than those in the earlier group.
6. There is doubtless substantial self-selection in the survey response, although the impact of this is unknown.

Despite these caveats, there is great value in examining the results in a descriptive fashion. These are without doubt the best data we have on long term impacts and student perceptions of the IDA intervention.

Outcomes

The survey measured attitudinal, job and educational outcomes for the two groups of students in addition to measuring the extent to which IDA changed the experience of the curriculum.

Curricular change

The IDA cohorts of students consistently report greater exposure to course content that included data analysis, use of computers, and working with tables and graphs. The most dramatic increase was in reports of analyzing data in class which increased from 49% to 67% for a percent change of 37%.  Clearly there were substantial changes in the curricular experiences between the two time periods. (Table 2).

One alternative explanation of change might be that students in the later time period were more likely to have taken a statistics course, whether at Lehman or elsewhere.  In fact, we found that students were slightly less likely to have taken a college level statistics course in the later period. (Table 3).  Given that the larger decline in statistics enrollment often occurred prior to students’ enrollment at Lehman, this most likely is a result of other changes at Lehman and at its main sending institutions rather than to changes in the sociology curriculum.

We were also interested in students’ perceptions of the role of computers and data analysis in the sociology curriculum. Comparing the two time periods, the percentage saying that the curriculum emphasized these things too little went from 29 to 18 (percent change -61) for statistics and data analysis and from 32 to 24 (percent change -33) for computer use. Only a very small percentage of alumni from either period felt that computer use and data analysis were emphasized too much. Clearly substantial numbers of alumni continue to feel that incorporation of data analysis and technology should be expanded.

Changes in perceptions of sociology as a discipline

One goal of the IDA approach to teaching sociology is to change students’ understandings of what the field of sociology consists of. A premise is that by actively “doing sociology” instead of simply reading about it, students will be more engaged with the field.  We asked students a series of items about their perceptions of what the purpose of sociology is (Table 6).  While two-thirds of the pre-IDA group said that all four items were “very important” less than half of the IDA group did so.  Alumni responses indicate that there was already strong agreement about the importance of research prior to the project, and this did not change.

However, there were other potentially important shifts. In particular, more recent alumni were less likely to agree that “developing theories of society” and “solving problems in society” were very important than were older alumni. The perception of sociology as focused on problem solving is generally perceived as problematic within the discipline because it is in opposition to a view of sociology as a scientific discipline focused on answering questions and testing hypotheses. Thus the decline of perceived importance of  “solving problems” is expected if the focus on sociology as a scientific discipline has grown and means that students have a more accurate understanding of the discipline. The decline in importance of theories is more problematic.  The social theory course (sociology 316) was the one course not substantially impacted by the IDA project, while at the same time there was a shift in the class in that in the later time period it was predominantly taught by a part time instructor. However, this result could also mean that data analysis has displaced theory in some courses.  This should be an area of further research about the sociology curriculum at Lehman.

Confidence with technology, data analysis and statistics

One goal of the IDA intervention is to increase students’ confidence in their abilities around technology, data analysis and statistics.  In this the changes are less clear and more difficult to interpret (Table 6). For the first item, confidence in interpreting tables and graphs, there was a modest increase. However, for the other three items, there were modest decreases in the proportion saying they were very confident. One partial explanation for this may be the previously discussed decline in enrollment in statistics classes during the same time period. Students who had not taken statistics were less confident in their abilities to calculate statistics and to interpret tables and graphs. (Table 7) More puzzling is the lower confidence in their ability to use a computer among the younger students. However, these differences were small and may also relate to work experience (see below).  Since some courses included data analysis work before IDA and some during the IDA period did not incorporate data analysis, self-reports of whether or not the student had analyzed data in at least one sociology class outside of 246 and 346 was also examined. Students who reported any data analysis experience were more likely to report being very confident and less likely to report being not very confident. (Experiencing data analysis in class was also highly correlated with in class computer use and in class work with graphs and tables so it is used as a proxy for all IDA approaches.)

Current jobs, computer use and education

Although it has been only a short time since the IDA cohorts have graduated, it is useful to compare some of their current work and educational experiences with earlier cohorts. However, it must be stressed that the difference of graduation period, i.e. the extra years of job and graduate school experience among the earlier graduates, likely explains many of the differences between the two groups.  Nonetheless, it can be concluded that technology, data analysis, writing and statistics are important in the careers of Lehman sociology graduates. For example, most alumni from all cohorts reported daily use of computers for one or more purposes.

Most alumni reported being employed for pay; this was somewhat higher among older cohorts. More than one third of alumni reported at least some graduate school enrollment, with a somewhat higher percentage among the older groups. More than two-thirds reported plans for future graduate work; younger cohorts were more likely to report such plans. The more recent alumni who had done graduate work were highly concentrated (more than 80%) in education programs. Students who reported having taken a statistics class and reported doing data analysis in a class besides 246 and 346 were more likely to have enrolled in graduate programs.

We asked about the importance of a variety of skills in work and graduate school. Writing and computer use were mentioned by overwhelming majorities of alumni as very important in both work and graduate school.  All of the skills were considered very important in students’ graduate programs. For students with jobs, the skills were considered “very important” by between a quarter and a third of alumni.

Qualitative Responses

Respondents were given an opportunity to share any thoughts they had about the sociology program:

In this last space, we invite you to give us any comments about your experiences at Lehman or since you left Lehman. Do you have any ideas that would help us to improve the sociology program

Almost all of the respondents provided qualitative comments, many of which touched on more than one topic.

The most common (n=18) type of response was a general positive comment about the Lehman sociology program.  Some examples are:

• My experience at Lehman was excellent. Lehman gave me to be proud of myself and be confident of what I am. I also advise my daughter to study sociology. She will finish in two semesters. My advise to the sociology dept is to continue the same education.
• I really enjoyed Lehman College and how the professors made themselves available.
• I had a very excellent experience at Lehman because things that I never know before I came to Lehman; staff and professors taught me. I would recommend anybody to study at Lehman because Lehman is a very excellent college to study.
• I miss Lehman and the helpful professors.
• My experience at Lehman was a positive one. I enjoyed very much all my sociology courses. The course topic themselves were interesting and all the professors were wonderful not only because of there personality but also because of the methods in which they used to educate us. I especially enjoyed the courses in which we used computers to research because that in addition to the standard textbook learning really helped in having us understand concepts and give us hands on experience by allowing us to do the research on our own.
• I will forever be indebted to the faculty for their dedication and interest in me as a student. … I don’t think I could’ve gotten a better sociology education anywhere else or with a better faculty.

The next most common response (n=12) was a suggestion that there be increased use of computers and research experiences. Typical comments are:

• Incorporate more programs on the computers using the latest programs such as excel, PowerPoint. Since finishing these programs are in great demand.
• I think that the use of more computers will enhance the sociology program.I would have to say that the sociology program is well structured but there should be more computers usage to explore society’s statistics and other material that relates to the class.
• I have had a great experience in majoring in Sociology at Lehman. Its given me an insight to the varied fields that I could go into. However, in regards to making calculations such as percentages, I wish I had more confidence and practice during my college years.
• Have more computer rooms, working computers and most important faster programs.

The next largest cluster of comments (n=8) focused on individual faculty members of courses.

Two other large clusters were the suggestion that there be an internship program (n=5) and the desire for more exposure to people (n=2) working in careers using sociology.

• Lehman sociology dept should offer more (internship) programs to their students. It should be for junior and senior students. The students would benefit more after graduation in the real world.
• Incorporate guest speakers currently working in a field of sociology. Many students remain uncertain to where this degree may lead.
• I suggest for Lehman to give the students enrolled an intern on sociology during their last year. It is very hard to get a job after one graduates.
• Graduating from Lehman with a BA in Sociology has not been easy. The same work study programs that are required for social work student needs to be given to sociology students.
• Sociology should have internships to get a better knowledge of their field.
• Students need to do intern, because once you graduate if the individual don’t have any experience they can’t get a job.
• As it stands now the program is good but could be fortified with mandatory internships-possibly eliminating the need for a minor.
• Maybe meet other experienced sociology researchers in their place of employment.

Other comments included two general negative comments (one expressed frustration at the alumna’s difficulty at finding a job while the other simply said “Re-evaluate your sociology professors!”), issues about registration or enrollment, and the need for more weekend or evening services. 

Conclusions

Implications for the future of IDA; implications for the sociology curriculum in general

Based on the alumni survey it seems clear that the IDA program has had a substantial impact on the experiences of Lehman sociology students. For sociology alumni the skills emphasized in the IDA model (such as data analysis, research, writing and interpretation of graphs) are important in their post-college careers and graduate education. Alumni from the IDA period have more confidence in their skills than older alumni as do students who self-reported participation in data analysis in classes.

What the program should consider doing in response to these findings:

1. Based on strong feelings of student satisfaction with the incorporation of data analysis and computer use, continue to develop and extend the use of innovative methods and materials to incorporate these experiences across the sociology curriculum.
2. Given the career and graduate education trajectories of our alumni, continue to develop a sociology curriculum that will allow them to be successful after college.
3. Think about the role of statistics in the curriculum. If students are no longer are likely to take statistics in a class outside of sociology, we may need to consider creation of a sociological statistics class that meets the needs of our students.  The importance of this is emphasized what appear to be long term consequences of the completion of a college statistics class.
4. Strengthen the place of theory throughout the curriculum.