Roger Kimball lists some things the Federal government could do without, specifically "the two National Endowments, the one for the Arts (so-called) and the one for the Humanities;" the Department of Education (especially after this); and the Department of Housing and Urban Development.
I actually disagree in part--the National Endowment for the Arts performs one useful function: insuring loaned art that would otherwise make traveling art exhibits impossibly expensive. But I endorse Kimball's other ideas.
Pre-blog, I once made this suggestion: there are 15 Cabinet-level departments of the Executive Branch, plus the EPA (which is nearly that rank), along with literally hundreds of "independent" agencies. Write the name of each department and agency--exempting only the Department of Defense--on a ping pong ball, and put all the balls in a randomizing lottery machine. Every September 30th (the end of the fiscal year), draw one ball--the agency listed gets zero-budgeted. If, within a year, if the majority of the electorate misses the agency, it's restored; if not, it's terminated. Regardless, next September 30th, draw another ball.
Impractical? Sure! But it's more "fiscally responsible" than Obama's budgets.