Chester put in a battling performance but were unable to absorb substantial Gateshead pressure as a half-time lead was overturned.
Craig Hobson put them ahead against the run of play on 28 minutes and despite being on the back foot, the Blues showed commendable grit to take the advantage into the break.
Gateshead dominated the second half, although Hobson did hit the post, and created numerous openings but looked on course for a frustrating afternoon.
But on 67 minutes Jon Shaw converted a debatable penalty to level and the Heed won the game with 10 minutes left when Jamie Chandler thumped home from close range.
Chester had been more down than up over the first five games, although performances on the road provided greater encouragement than those on home soil.
Having watched his side battle back to earn a 2-2 draw at Kidderminster Harriers on Monday, manager Steve Burr made three changes to his starting side.
Jadan Hall came straight in having been borrowed from Burnley and Jon Worsnop returned in goal while Ben Heneghan replaced captain Matty Brown in an unexpected change.
Gateshead went close to promotion last season, losing out to Cambridge United in the play-off final, and were fancied to be there or thereabouts again.
Gary Mills had to do without Marcus Maddison, sold to Peterbrough United midweek, but signed ex-Wrexham centre forward Danny Wright the night before the game.
Other than an odd 6-1 home loss to Grimsby Town, the sixth-placed Heed had made a good start and won 1-0 at Altrincham on Monday.
Mills made two changes to his side with keeper Adam Bartlett returning after two games out and Wright replacing Maddison.
Having conceded five times inside 10 minutes this season, the Blues would have been quite pleased with the quiet start to the game.
John Rooney had a shot blocked on four minutes and Wright sent a free header wide eight minutes later but that was about all the excitement the opening quarter offered.
Gateshead often dominate the possession stakes and began to dictate things as the half wore on, although without creating too much.
Chester were content to allow them to have the ball and went ahead on 28 minutes when Hobson punished a defensive howler.
Michael Kay knocked the ball forward and centre backs Ben Clark and James Curtis collided when attempting to mop up, allowing Hobson to fire under Bartlett from 10 yards and put the Blues into the lead for the first time this season.
Gateshead responded well with left winger Alex Rodman looking dangerous, skinning Danny Taylor but missing everyone with his cutback and then blasting straight at Worsnop after John Oster teed him up.
Hobson dragged a shot into the side netting after a smart Rooney pass for Chester but the Heed continued to push hard.
Some wholehearted defending was required to keep them out but the Blues stood strong and took a one-goal lead into the break.
Gateshead were sent out early for the second half but could have been two down on 50 minutes when Hobson met an excellent Taylor cross and his header beat Bartlett but rebounded off the post.
That shocked the home side into action and Worsnop made a smart point-black stop to prevent Chandler finishing off Shaw's knock-down on 55 minutes.
Seconds later and Worsnop was beaten but Michael Kay was on hand to clear Wright's shot off the line before Shaw couldn't connect with a low cross from sub Rob Ramshaw.
Burr replaced Rooney with Wayne Riley, so impressive off the bench at Kidderminster, but the hosts kept on coming with Clark hooking over after Curtis' shot rebounded off Wright.
Gateshead equalised on 67 minutes when referee Jason Whiteley ruled Danny Harrison impeded Shaw from a corner and awarded a penalty.
It was a decision that could have gone either and Shaw's powerful spot kick squeezed past Worsnop to level the scores.
Mr Whiteley could have pointed to the spot again minutes later when Heneghan appeared to bring down Rodman but the referee turned strong appeals down and Kay made a fine block to prevent Shaw getting a second moving into the closing stages.
Near constant pressure told on 80 minutes when Ramshaw found fellow sub Lewis Guy and he pulled the ball back for Chandler to smash into the roof of the net to give Gateshead the lead.
Chester couldn't find a response in the final few minutes and the home side saw the game out, leaving the Blues to focus on back-to-back home games against Macclesfield Town and Torquay United.
Chester FC: Worsnop, Taylor, Kay, Heneghan, G Roberts, Harrison, Rooney (Riley 62), James (Iwelumo 87), McConville, Hobson (Winn 77), Hall.
Subs: Brown, C Roberts.
Goal: Hobson 28.
Booked: G Roberts, Hobson.
Gateshead: Bartlett, Jones, Curtis, Clark, O'Donnell, Oster (Ramshaw 52), Chandler, Turnbull, Wright (Guy 77), Shaw, Rodman.
Subs: Brown, Allan, Baxter.
Goals: Shaw 67 (pen), Chandler 80.
Referee: Jason Whiteley (Leeds).
Attendance: 1,138.
