ElasticSearch: Difference between revisions
No edit summary |
(→Memory) |
||
| (9 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
Up to: [[Part Of::Tech WG]], [[Part Of::Search]] | |||
== Config == | == Config == | ||
| Line 10: | Line 12: | ||
</pre> | </pre> | ||
<code>/home/mastodon/.env.production</code> | |||
<pre> | |||
idk if we should put this part online but there is some config here too | |||
</pre> | |||
=== Service === | |||
Use the default service, except we also make it restart when it dies: | |||
<code>/lib/systemd/system/elasticsearch.service</code> | |||
<syntaxhighlight lang="ini"> | |||
[Service] | |||
# Other stuff.... | |||
ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet | |||
Restart=always | |||
RestartSec=30 | |||
# ... | |||
</syntaxhighlight> | |||
=== Memory === | |||
[https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html the ES docs] say to allocate half your available memory to the heap. At the moment (23-10-26) we are using 2.5G/8G memory at rest. Going to try allocating 2GB and see what happens | |||
'''Notes:''' | |||
* Keep getting OOM, so changing 3GB to 2GB. Search performance seems fine, indexing jobs finish, etc. This will probably be fine for now unless we have some huge spike in users/searchers - jonny, 23-11-25 | |||
<code>/etc/elasticsearch/jvm.options.d/heap.options</code> | |||
<pre> | |||
-Xms2g | |||
-Xmx2g | |||
</pre> | |||
== Creds havers == | |||
* [[Jonny]] | |||
* [[Lina]] | |||
== References == | == References == | ||
| Line 16: | Line 55: | ||
Docs: | Docs: | ||
* https://docs.joinmastodon.org/admin/elasticsearch/ | * https://docs.joinmastodon.org/admin/elasticsearch/ | ||
* THIS is actually the current docs, the other urls that are EXACTLY THE SAME are no longer maintained - https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#set-jvm-heap-size | |||
* https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html | |||
* https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html | |||
* https://blog.thms.uk/2023/09/mastodon-elasticsearch | * https://blog.thms.uk/2023/09/mastodon-elasticsearch | ||
* (pretty old at this point) https://github.com/felx/mastodon-documentation/blob/master/Running-Mastodon/Elasticsearch-guide.md | * (pretty old at this point and also ES says not do to the JVM memory size thing) https://github.com/felx/mastodon-documentation/blob/master/Running-Mastodon/Elasticsearch-guide.md | ||
Since we are running on a small VPS, we might need to configure it for a low memory environment! | Since we are running on a small VPS, we might need to configure it for a low memory environment! | ||
== Discord == | == See also == | ||
* [[Search]] | |||
== History == | |||
=== Threads === | |||
* Initial enabling with [[Jonny]] and [[Lina]] - https://discord.com/channels/1049136631065628772/1052452072806686751/1167287930742652958 | |||
=== Discord === | |||
{{Message | {{Message | ||
| Line 31: | Line 84: | ||
|Link=https://discord.com/channels/1049136631065628772/1167287930742652958/1167289017205792969 | |Link=https://discord.com/channels/1049136631065628772/1167287930742652958/1167289017205792969 | ||
}} | }} | ||
[[Category:Search]] | |||
[[Category:Config]] | |||
[[Category:Services]] | |||
Latest revision as of 01:54, 26 November 2023
Config
List any changed config values here!!!
/etc/elasticsearch/elasticsearch.yml:
network.host: localhost discovery.type: single-node xpack.security.enabled: true
/home/mastodon/.env.production
idk if we should put this part online but there is some config here too
Service
Use the default service, except we also make it restart when it dies:
/lib/systemd/system/elasticsearch.service
[Service]
# Other stuff....
ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet
Restart=always
RestartSec=30
# ...
Memory
the ES docs say to allocate half your available memory to the heap. At the moment (23-10-26) we are using 2.5G/8G memory at rest. Going to try allocating 2GB and see what happens
Notes:
- Keep getting OOM, so changing 3GB to 2GB. Search performance seems fine, indexing jobs finish, etc. This will probably be fine for now unless we have some huge spike in users/searchers - jonny, 23-11-25
/etc/elasticsearch/jvm.options.d/heap.options
-Xms2g -Xmx2g
Creds havers
References
Docs:
- https://docs.joinmastodon.org/admin/elasticsearch/
- THIS is actually the current docs, the other urls that are EXACTLY THE SAME are no longer maintained - https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html#set-jvm-heap-size
- https://www.elastic.co/guide/en/elasticsearch/guide/current/hardware.html
- https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html
- https://blog.thms.uk/2023/09/mastodon-elasticsearch
- (pretty old at this point and also ES says not do to the JVM memory size thing) https://github.com/felx/mastodon-documentation/blob/master/Running-Mastodon/Elasticsearch-guide.md
Since we are running on a small VPS, we might need to configure it for a low memory environment!
See also
History
Threads
- Initial enabling with Jonny and Lina - https://discord.com/channels/1049136631065628772/1052452072806686751/1167287930742652958
Discord
OK yet again starting thread on getting ElasticSearch running