Brendan Rodgers' men made it back-to-back wins in the league for the first time this season and leapfrogged above the Hammers - but they had to come from behind late in the game to do so.

Glen Johnson put them into the lead after just 10 minutes when he produced a goal-of-the-season contender from the edge of Jussi Jaaskelainen's penalty area.

Mark Noble slotted from the spot after Joe Allen was adjudged to have handled in the penalty area before Steven Gerrard turned a Matt Jarvis cross past his own 'keeper to give West Ham the lead.

However, Joe Cole struck against his old club to equalise and then Collins turned a Jordan Henderson cross  into his own net to send the travelling Reds into euphoria.

All the build-up to the clash revolved around the absence of Luis Suarez, who was ruled out after picking up his fifth yellow card against Southampton, but his replacement Shelvey proved to be the hero of the hour.

Upton Park was on its feet on six minutes when Jarvis's cross was scrambled clear by Daniel Agger and Mohamed Diame fired the loose ball against Carlton Cole, who deflected it a yard wide of Pepe Reina's post.

The Hammers posed a real threat down the left flank with Jarvis, who was constantly keen to spin a deadly cross in behind Martin Skrtel and Agger and this would prove to be a real thorn in the Reds' side throughout the match.

But first, Johnson fired the visitors ahead on 10 minutes. The full-back controlled a Gerrard pass, nudged the ball away from Jarvis, sped towards the edge of the box and then hammered the ball beyond Jaaskelainen and into the top corner of the goal behind.

The strike really ought to have been followed by the most exuberant of celebrations, such was its sheer quality; however, the former Hammer remained rooted to the spot from where he beat Jaaskelainen and refused to celebrate as a mark of respect to his old club.

Sterling should have put two more past West Ham in the minutes that followed. His first chance he carved through his own volition as he cut in and fired low, his second came as the result of more artistry from the simply irrepressible Johnson.

But the threat down the Hammers' left wing lingered on despite Liverpool's early dominance and just before half-hour mark, Jarvis arrowed the ball across the face of goal to where Cole was poised to convert but Agger launched himself in the way to make an outstanding block.

And then disaster struck for Liverpool. The referee adjudged Joe Allen to have handled the ball in the area and Noble stepped up and slotted the penalty low to Reina's right.

West Ham applied more pressure and minutes before half-time they stole ahead. Noble fed Jarvis from a briskly-taken free-kick and the winger sent another tantalising cross into the middle, where Gerrard rose in a vain attempt to clear the danger but turned the ball past Reina instead.

West Ham started the second half just as they finished the first - full of confidence and menace. Diame stood out in the middle of the park and ahead of him Kevin Nolan forged a couple of decent goalscoring opportunities inside the Liverpool penalty area.

Noble and Matthew Taylor interchanged behind Carlton Cole, who led the line brilliantly.

However, Sterling went agonisingly close at the other end after Stewart Downing picked him out unmarked inside the box and it took a superb save from Jaaskelainen to deny the 18-year-old. Downing then found Shelvey with a fine cross but the makeshift striker could only nod wide.

With 15 minutes remaining, Sterling and Jordan Henderson exchanged passes before the ball was slid into the path of Cole, who controlled and smashed the ball past Jaaskelainen to equalise.

Then Henderson broke down the right and crossed into the middle, where Shelvey was waiting - but Collins got there first only to flick the ball into his own net and hand the Reds a memorable win.

Liverpoolfc.com journalists' man of the match: Glen Johnson.