William Clyde Allen III, 39, was taken into custody in in the small northern Utah city of Logan, said Melodie Rydalch, a spokeswoman for the state's U.S. attorney's office.
The arrest comes after authorities confirmed an investigation into two envelopes once thought to contain ricin and later found to be castor seeds, the substance from which the poison is derived. They can cause injury if swallowed.
The FBI said there were potentially hazardous chemicals involved with their operation in Utah on Wednesday, but declined to give additional details. No attorney was immediately listed for Allen.
The envelopes addressed to Defense Secretary Jim Mattis and the Navy's top officer, Adm. John Richardson, were isolated at a mail screening facility and sent to the FBI. No one was injured, and neither envelope entered the Pentagon. Mattis is traveling in Europe this week.
An envelope also was sent to the president Monday with unknown contents. The Secret Service says it didn't reach the White House.
Allen served in the Navy from 1998 to 2002, according to Navy records. He worked as a damage control fireman apprentice.
Two years after his Navy tenure ended, he was charged in a child sex-abuse case involving two girls he had an unspecified "relationship of trust" with, court documents state. He later pleaded guilty to lesser neglect and abuse charges and did not have to register as a sex offender.
That same year, a woman filed a protective order against him in a separate case. He disputed her allegations, the details of which are not public, but agreed to the protective order.
In 2008, he pleaded guilty to attempted aggravated assault and served about 2 ½ years in prison, authorities said. He was released in 2011.
New charges are expected to be filed later this week.
Associated Press writer Brady McCombs contributed to this story.