A proposal under consideration by the agency would generally require brokers to route small investors’ market orders into auctions, where trading firms would compete to execute them, people familiar with the matter said. SEC Chairman

Gary Gensler

first floated the idea of such auctions in June. The auction requirement would apply to orders less than $200,000 by customers who average fewer than 40 trades a day, the people said. 

Brokers would have a way out. Instead of sending the orders to auctions, the brokers could attempt to have them filled at the midpoint price or better, the people said. Such a requirement would effectively push brokers to try harder to deliver midpoint prices for their customers.

The ideas are part of a package of proposals to revamp stock-market structure that the SEC is expected to roll out in the coming months. Mr. Gensler called for the changes after last year’s trading frenzy in

GameStop Corp.

and other meme stocks, saying he wanted to promote greater efficiency and transparency in the market. 

The proposed midpoint requirement and auctions would apply to market orders. Commonly used by small investors, market orders are instructions entered through a brokerage to buy or sell stocks at whatever their current market price is.

Many traders view the midpoint as a fair price to get on a market order because it involves a buyer and seller meeting halfway between the so-called bid and ask price. For instance, if shares of

Tesla Inc.

can be purchased on stock exchanges for $225.55 and sold for $225.45, the midpoint would be $225.50. 

Currently, brokerages such as

Robinhood Markets Inc.

and

Charles Schwab Corp.

send most retail market orders to a handful of giant electronic trading firms such as Citadel Securities and

Virtu Financial Inc.

These trading firms, known as wholesalers, typically execute those orders somewhere between the bid and ask price. 

Wholesalers often pay brokers for the right to handle investors’ orders—a controversial practice called payment for order flow. They earn a profit for buying shares for slightly less than they sell them for. Wholesalers might execute an investor’s order at the midpoint, but there is no direct requirement for the broker to seek such a price.

Mr. Gensler has also said he wants the SEC to introduce a “best execution” rule, which would toughen brokers’ obligations to seek out the best available prices for their customers. He has pledged to tackle several other areas, including “tick sizes”—the price increments in which stocks are quoted and traded—and disclosures of payment for order flow.

SEC officials say the changes could be proposed in batches, rather than all at once. They would then undergo a lengthy review and incorporate public feedback before being codified into agency regulations. 

Wall Street lobbying groups, electronic-trading firms and brokerages are likely to push back against any significant rule changes. Some executives say the industry will likely sue in court to block the SEC’s proposed regulations—meaning the proposals could be years away from becoming reality, if ever.

Most big brokerages and trading firms say the current system works well because it allows individual investors to get swift executions at prices better than those posted on exchanges. Industry data show that these improved prices result in billions of dollars of savings for investors every year.

Some analysts and industry veterans critical of current practices complain that brokerages are failing to check to see if their customers’ orders could be filled at the midpoint—a missed opportunity for their customers. 

A number of exchanges and off-exchange trading platforms offer midpoint-trading mechanisms that allow buyers and sellers to meet halfway between the bid and the ask price. The SEC’s proposal is aimed at pushing brokers to scour the marketplace for such midpoint opportunities, instead of sending orders to electronic-trading firms willing to pay for the order flow.

Brokerages and trading firms counter that small investors often get midpoint executions under the current system. Schwab has said that it delivers midpoint or better prices on more than 50% of market orders.

Still, brokerages are uneven in their ability to deliver midpoint prices, according to an academic study released in August based on 85,000 stock trades executed at a half-dozen popular brokerages over six months.

The study’s authors found that at Schwab’s TD Ameritrade, they got midpoint prices or better on 69% of their trades, while at Robinhood the comparable rate was 31%, according to data from Christopher Schwarz, a finance professor at the University of California, Irvine, and the lead author of the study.

Write to Alexander Osipovich at alexo@wsj.com and Paul Kiernan at paul.kiernan@wsj.com