If there are still 3 Nephites walking the earth today, this means the priesthood was never taken from the earth and there was no need for Joseph Smith to restore it.
The Three Nephites are not the only priesthood holders who remained on the earth during the great apostasy that followed the Savior’s mortal ministry. I’m sure you must agree that John the Beloved, who was banished to Isle of Patmos, was promised by the Savior that he could remain in mortality on the earth until the Savior would come to initiate his great Millennial reign (See John 21:22-23).
Not only that, but the lost ten tribes, whom the Savior visited following his resurrection, also undoubtedly have the priesthood with them.
Now all of these priesthood authorities have one thing in common that would prevent them from restoring the Savior’s Church following the great apostasy. Not only are they hidden from the world, but they did not have the keys of authority necessary to restore anything,
The angel Moroni revealed the Book of Mormon to Joseph Smith because he was the last author of that sacred volume, and hid it up in the earth for the very purpose of it being revealed to Joseph Smith at the proper time.
John the Baptist, who held the keys of the Aaronic Priesthood in his day, and who baptized the Savior, appeared to Joseph Smith and Oliver Cowdery on May 15, 1829, and restored to them the Aaronic Priesthood.. In doing so, John the Baptist acted under the priesthood authority of Peter, James and John, who held in their day the keys of the Melchizedek Priesthood, and he advised them that Peter, James and John would soon visit them to ordain them to the Melchizedek Priesthood, which they did sometime in June of that same year. The Melchizedek Priesthood gave Joseph and Oliver the requisite authority to act in the name of the Savior to perform all functions associated with the restoration of the Gospel of Jesus Christ and the organization of his Church and Kingdom. All of this was done in fulfilment of prophecies from both the Old and New Testaments, as well as the Book of Mormon.
On 3 April, 1836, at the time of the dedication of the holy temple that the saints had built to the Lord in Kirtland, Ohio, Jesus Christ himself appeared to Joseph Smith and Oliver Cowdery, accepting the temple as his House. After this marvelous vision, Moses, Elias and Elijah appeared in turn, each committing to Joseph and Oliver the keys of the Priesthood that they held in their dispensations. Moses committed unto us the keys of the gathering of Israel from the four parts of the earth, and the leading of the ten tribes from the land of the north. Elias committed the dispensation of the gospel of Abraham, saying that in us and our seed all generations after us should be blessed. And then Elijah the prophet, who was taken to heaven without tasting death, stood before us, and said: Behold, the time has fully come, which was spoken of by the mouth of Malachi–testifying that he [Elijah] should be sent, before the great and dreadful day of the Lord come–To turn the hearts of the fathers to the children, and the children to the fathers, lest the whole earth be smitten with a curse–Therefore, the keys of this dispensation are committed into your hands; and by this ye may know that the great and dreadful day of the Lord is near, even at the doors (Doctrine and Covenants, Section 110).
It is interesting to note that before the visit of Elijah very little interest in genealogy had existed anywhere except to establish the lines of kingly authority. Rather immediately after his visit the British Civil Registry was established, and birth, marriage and death records where collected and stored there beginning in 1837. Also, in 1836 the Boston Genealogical Society, the first such society to be organized in the Unites States, was founded. The “spirit of Elijah” or the interest in searching out ancestry so that their temple work can be performed in the Lord’s holy temples, began to spread across the earth beginning with his visit to the Kirtland Temple.