Attorney General Eric Holder is changing how the Justice Department prosecutes non-violent drug offenders.
Instead of facing a mandatory minimum prison sentence, they would be sent to drug treatment and community service programs.
Holder said overhauling the federal prison policy will improve sentencing policies across the country.
Under the altered policy, Holder said defendants will be charged with sentences that "are better suited to their individual conduct, rather than excessive prison terms more appropriate for violent criminals or drug kingpins."
The attorney general is calling his new approach the Smart On Crime Initiative, where federal prosecutors across the country are encouraged to create local guidelines "for determining when federal charges should be filed, and when they should not," according to the Associated Press.
"By targeting the most serious offenses, prosecuting the most dangerous criminals, directing assistance to crime 'hot spots,' and pursuing new ways to promote public safety, deterrence, efficiency, and fairness - we can become both smarter and tougher on crime," Holder said.
The United States imprisons a higher percentage of its population than other major countries, largely because of anti-drug laws passed in the 80s and 90s.
Now federal prisons are operating nearly 40 percent above capacity.
"We cannot simply prosecute or incarcerate our way to becoming a safer nation," Holder said.
"Today, a vicious cycle of poverty, criminality and incarceration traps too many Americans and weakens too many communities," he said. "However, many aspects of our criminal justice system may actually exacerbate this problem, rather than alleviate it."