Yes... especially if the IRS is expecting a return (based on the letter received after applying for the FEIN).

AND, those expenses may or may not be immediately deductible.  Check into the need to capitalized as start-up costs, depending on the fact pattern.