More than 50 far-right vigilantes were armed with pepper spray, a bayonet, a machete and batons, police said.
The group is believed to support the Third Way, a far-right party with suspected links to neo-Nazi groups.
The party has called on its members to take action against border crossings, which have increased since August.
Police seized weapons and made the group leave the area near the town of Guben on Saturday night and early on Sunday, a spokesperson said.
A counter-demonstration was held in opposition to the far-right patrols.
Germany has deployed an extra 800 police officers to the Polish border in an effort to control the number of migrants attempting to enter the EU from Belarus, the interior minister was quoted as saying on Sunday.
There have been more than 6,000 unauthorised entries into Germany from Belarus and Poland this year, he said.
There are no plans to shut the border with Poland, he said, but added that controls may need to be reintroduced.