When trying to index a WordPress blog on Naver, you will inevitably run into a file called robots.txt.
Particularly after learning that Naver’s search crawler is named Yeti, you might wonder: “Do I absolutely have to add Yeti to my robots.txt?”
If you check the robots.txt file of a standard WordPress site, it typically looks like this:
Plaintext
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://newspaperioccasionallyread.com/sitemap.xml
To understand whether this configuration is correct and whether you need to add User-agent: Yeti, you first need to understand what robots.txt actually is.
What is robots.txt?
robots.txt is a text file located at the root directory of a website.
It acts as an instruction manual for search engine crawlers (search robots), telling them which areas of your website they are allowed or forbidden to crawl.
Here, crawling refers to the act of a search robot accessing a web page to read its content.
Naver Search names its crawler Yeti.
Therefore, if you are aiming for visibility on Naver, ensuring your robots.txt is configured without errors is essential.
What is User-agent?
User-agent is the very first directive specified in a robots.txt file.
For instance, consider the following rule:
Plaintext
User-agent: *
The asterisk (*) serves as a wildcard representing all search crawlers, rather than targetting a single specific bot.
Conversely, you can explicitly target a specific bot by name:
Plaintext
User-agent: Yeti
In this case, the declared rules apply exclusively to Naver’s Yeti crawler.
Do You Really Need to Declare Yeti Separately?
For a standard WordPress site, you do not necessarily need to do so.
If you use User-agent: * and grant access to your main content areas, Yeti will naturally adhere to those global permissions.
Naver also explicitly states in its documentation that site owners can choose between defining universal access rules for all bots or specifying targeted directives for individual crawlers.
The critical factor is not whether the word “Yeti” appears in the file, but whether the crawler can actually access the pages necessary for indexing.
Understanding Allow and Disallow
To grasp robots.txt mechanics, you must understand Allow and Disallow.
Disallow is a rule that blocks crawlers from accessing specified paths.
For example:
Plaintext
Disallow: /wp-admin/
This directive restricts crawlers from accessing the /wp-admin/ directory.
Allow, on the other hand, explicitly permits access to specific sub-paths within a restricted directory.
The standard WordPress setup shown below blocks general admin area access while explicitly permitting AJAX requests essential for site functionality:
Plaintext
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Why Block the WordPress Admin Area?
The WordPress admin dashboard is not public content intended for search engine users.
It contains sensitive administrative functions, core backend utilities, and publishing interfaces.
Consequently, indexing these pages serves no SEO purpose.
However, accidentally blocking access to actual public content—such as files inside wp-content or regular post URLs—can severely disrupt indexing.
When modifying robots.txt, restrict only internal system areas that hold no value for search engines.
The Most Dangerous robots.txt Configuration
The following rule poses a critical risk to any site seeking search engine visibility:
Plaintext
User-agent: *
Disallow: /
This code tells search engines to block the entire website from being crawled.
If applied to a WordPress site meant for Naver indexing, search crawlers will be completely barred from reading your content.
Always verify that a global block like Disallow: / is not accidentally deployed.
Why Declare a Sitemap in robots.txt?
You can specify the location of your XML sitemap directly inside robots.txt:
Plaintext
Sitemap: https://newspaperioccasionallyread.com/sitemap.xml
This informs crawlers precisely where to locate your sitemap index.
A sitemap contains structured lists of your site’s main URLs, helping search engines discover your site hierarchy efficiently.
However, adding a sitemap does not guarantee immediate inclusion in search results; it merely guides crawler navigation rather than enforcing indexing.
robots.txt vs. Sitemap: Key Differences
It is crucial not to confuse these related search tools:
robots.txt: Defines crawling permissions and access rules for search bots.
Sitemap: Directs crawlers to essential site URLs and architectural structure.
Crawl Request: Submits specific URLs directly to search bots for indexing inspection.
Indexing: The process where collected content is processed and stored in search engine databases for query display.
While interconnected, these functions serve entirely distinct purposes.
How to Verify Your robots.txt File
After making edits, verify the changes live by opening the file directly in your web browser.
If your WordPress domain is [https://newspaperioccasionallyread.com](https://newspaperioccasionallyread.com), enter:
[https://newspaperioccasionallyread.com/robots.txt](https://newspaperioccasionallyread.com/robots.txt)
This loads the live file served directly by your web host.
What matters is the public file served to external visitors and crawlers, not just the text draft saved inside your WordPress dashboard.
Managing File Structure on Cafe24 Hosting
If you host WordPress on Cafe24 and manually manage your robots.txt, access your server via FTP software such as FileZilla.
Navigate to the root directory where core WordPress files reside:
Plaintext
wp-admin
wp-content
wp-includes
index.php
wp-config.php
If robots.txt exists as a physical file, place it directly inside this main root folder.
Naver requires robots.txt to reside strictly at the site’s root directory to be recognized.
Conclusion
To rank a WordPress site on Naver, explicitly adding the string “Yeti” to robots.txt is not required.
The priority is constructing a clean, permission-friendly robots.txt file that allows Naver crawlers unobstructed access to your public pages.
A standard baseline configuration for most blogs:
Plaintext
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: https://newspaperioccasionallyread.com/sitemap.xml
After deploying, verify the live URL in your browser and run a diagnostic check using Naver Search Advisor.
Keep in mind that establishing a valid robots.txt will not yield immediate search rankings on its own. robots.txt serves strictly as an access control framework, not a direct ranking mechanism.