Without any guidelines from Google, page-load speed is subjective. Providing you use tools such as Google's page speed or webpagetest (http://www.webpagetest.org) to assess whether you have an issue and then optimise as much as you can, you should be on track.

It's well known that visitors bail if a page doesn't load quickly enough but its not often a priority fix unless it's really bad; it makes sense that Google is making this an SEO issue as a sloooow loading page will mean a poor search-experience for the user.