I have virtually zero experience with a rifleAmmo is certainly a factor but so is the rifle. The rifle is just as important as the ammo. Not knowing what your experience is so I don't know how to enplane why. A rifle can be just as inconsistent as the ammo.
A bullet that is wavering because of the gun will become more and more unsteady the farther it goes. If it's off by a quarter inch at 25 yards, should mean it will be off by 3 times that at 100 yards. However that could change because of the air flow over and around the bullet, making it be off even more the the 3 times what one would expect.
You are correct in that it is mostly the ammo. However the muzzle crown can make bullets be inconsistent and create variables that can't be predicted or consistent. Defects in the chamber, riffling, and muzzle crown can make one bullet do something another wont. The farther the distance the more or less it may effect the trajectory.
But a fair amount of shooting experience with a pistol
I mostly shoot at 7 yards to 10 meters with a pistol and can pretty well tell how accurate the "system" of the gun the ammo and the shooter is at those distances
I can kind of tell what the more about the accuracy of the gun by using a fixture to rest the gun to shoot it
But without doing that I assume most of the variance in my shots is caused by the shooter and not the gun
I would assume the same is true when I shoot at 25 yards and if I shoot from a fixture the gun is pretty much still as accurate at 25 yards as it is at 10 meters.
So there is really no reason that I need to shoot at 25 yards to test the gun but only to text my skills as a shooter
I would assume the same is true when you shoot a rifle at 100 yards using a scope as opposed to shooting at 25 yard using iron sights
I think a scop is probably useful mostly when you are trying to overcome your ability as a shooter to overcome your variability in sighting in a target and long distances and really nothing to do with the rifle's accuracy