With so many millions of the Silence on Earth Prime-Time, there were probably plenty still alive by 1980 - despite the Doctor's post-hypnotic suggestion to all the inhabitants of the planet to kill the aliens on sight. Therefore, the Silence would have still been around when the Galactican fleet arrived. With their numbers greatly reduced, the Silence were unable to prevent the Galacticans from intermingling with their Terran cousins. They couldn't prevent those Galacticans from cannibalizing the spaceships of that ragtag fleet to give a boost to the technology of Earth.
It certainly splains how the communications and computer software industries (of Toobworld, at least) had such an incredible leap in development after 1980.