But those most concerned about losing work, including those who already have, are the most likely to say the U.S. is headed for a recession, or even depression.