225 Commits

Author SHA1 Message Date
wxiaoguang
58d2a87c6c update go&js dependencies (#34262) 2025-04-23 21:22:40 +00:00
Lunny Xiao
ae0af8ea5b Refactor Git Attribute & performance optimization (#34154)
This PR moved git attributes related code to `modules/git/attribute` sub
package and moved language stats related code to
`modules/git/languagestats` sub package to make it easier to maintain.

And it also introduced a performance improvement which use the `git
check-attr --source` which can be run in a bare git repository so that
we don't need to create a git index file. The new parameter need a git
version >= 2.40 . If git version less than 2.40, it will fall back to
previous implementation.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: yp05327 <576951401@qq.com>
2025-04-11 21:41:29 +08:00
Thomas E Lackey
fa49cd719f feat: Add sorting by exclusive labels (issue priority) (#33206)
Fix #2616

This PR adds a new sort option for exclusive labels.

For exclusive labels, a new property is exposed called "order", while in
the UI options are populated automatically in the `Sort` column (see
screenshot below) for each exclusive label scope.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-10 17:18:07 +00:00
TheFox0x7
ee3c82f874 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
TheFox0x7
0fde8ecd55 Enable testifylint rules (#34075)
enable testifylint rules disabled in:
https://github.com/go-gitea/gitea/pull/34054
2025-03-31 01:53:48 -04:00
TheFox0x7
2a59dfbd47 enable staticcheck QFxxxx rules (#34064) 2025-03-29 17:32:28 -04:00
Lunny Xiao
c1b9ecca34 Decouple Batch from git.Repository to simplify usage without requiring the creation of a Repository struct. (#34001)
No logic change
2025-03-27 17:35:48 +00:00
wxiaoguang
8745129c9c Fix incorrect code search indexer options (#33992)
Fix #33798

Co-authored-by: Giteabot <teabot@gitea.io>
2025-03-24 17:18:21 +01:00
charles
b956cee06f Fix file name could not be searched if the file was not a text file when using the Bleve indexer (#33959)
Close #33828

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 19:00:02 +00:00
Andreas Svanberg
0da7318cf3 Allow filtering issues by any assignee (#33343)
This is the opposite of the "No assignee" filter, it will match all
issues that have at least one assignee.

Before
![Before
change](https://github.com/user-attachments/assets/4aea194b-9add-4a84-8d6b-61bfd8d9e58e)

After
![After change with any
filter](https://github.com/user-attachments/assets/99f1205d-ba9f-4a0a-a60b-cc1a0c0823fe)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 04:25:36 +00:00
Lunny Xiao
c88e71c1d2 Refactor functions to reduce repopath expose (#33892) 2025-03-16 03:14:56 +00:00
wxiaoguang
a0e0a30d23 Make SearchMode have default value and add comments (#33863)
* Make `SearchMode` have default value if it is empty
* Add some comments for the "match" queries
* Fix a copy-paste mistake in `buildMatchQuery` (`db.go`)
* Add missing `q.Analyzer = repoIndexerAnalyzer`, it is in old code,
although I do not see real difference ....
2025-03-14 18:06:31 +00:00
wxiaoguang
403775e74e Improve issue & code search (#33860)
Each "indexer" should provide the "search modes" they support by
themselves. And we need to remove the "fuzzy" search for code.
2025-03-13 11:07:48 +08:00
TheFox0x7
ae3a18e01a Remove context from git struct (#33793)
Argument is moved from struct init in command run, which lets us remove
context from struct.
2025-03-04 11:56:11 -08:00
TheFox0x7
cc1fdc84ca Use test context in tests and new loop system in benchmarks (#33648)
Replace all contexts in tests with go1.24 t.Context()

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-20 09:57:40 +00:00
Lunny Xiao
69de5a65c2 Fix project issues list and counting (#33594)
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-17 05:14:56 +00:00
Darren Hoo
950abfe8ee enable literal string for code search (#33590)
Close: #33588

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2025-02-16 11:28:06 +01:00
Lunny Xiao
6999651b6d Fix unnecessary comment when moving issue on the same project column (#33496)
Fix #33482
2025-02-05 11:51:10 +08:00
TheFox0x7
2a02734f93 Refactor older tests to use testify (#33140)
Refactor checks to use assert/require
Use require.Eventually for waiting in elastic and meilisearch tests
Use require to exit early instead of assert
2025-01-09 09:21:47 +08:00
wxiaoguang
45973a100b Fix bleve fuzziness search (#33078)
Close #31565
2025-01-03 00:32:02 +08:00
TheFox0x7
33e8e82c4b Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
silverwind
1cfb718976 Update golangci-lint to v1.62.2, fix issues (#32845)
Update it and fix new issues related to `redefines-builtin-id`
2024-12-15 02:31:07 +00:00
yp05327
d43620e7bc Add is_archived option for issue indexer (#32735)
Try to fix #32697

Reason:
`is_archived` is already defined in the query options, but it is not
implemented in the indexer.
2024-12-12 07:33:31 +08:00
wxiaoguang
e619384098 Add label/author/assignee filters to the user/org home issue list (#32779)
Replace #26661, fix #25979

Not perfect, but usable and much better than before. Since it is quite
complex, I am not quite sure whether there would be any regression, if
any, I will fix in first time.

I have tested the related pages many times: issue list, milestone issue
list, project view, user issue list, org issue list.
2024-12-11 06:33:24 +00:00
wxiaoguang
44909f6e2c Fix markup render regression and fix some tests (#32640)
Fix #32639,
https://github.com/go-gitea/gitea/issues/32608#issuecomment-2497918210

By the way, fix some incorrect SQLs (use single quote but not double
quote)
2024-11-26 03:04:55 +08:00
Rowan Bohde
68731c07c5 Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
Bruno Sofiato
f64fbd9b74 Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-11-06 20:51:20 +00:00
wxiaoguang
5e6523aa57 Update go dependencies (#32389) 2024-10-31 12:05:54 +00:00
Bruno Sofiato
900ac62251 Allow code search by filename (#32210)
This is a large and complex PR, so let me explain in detail its changes.

First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).

I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.

I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.

The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).


![image](https://github.com/user-attachments/assets/9d938d86-1a8d-4f89-8644-1921a473e858)

Resolves #32096

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-11 23:35:04 +00:00
Bruno Sofiato
d266d190bd Fixed race condition when deleting documents by repoId in ElasticSearch (#32185)
Resolves #32184

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-03 16:03:36 +00:00
Bruno Sofiato
99d0510cb6 Change the code search to sort results by relevance (#32134)
Resolves #32129

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-09-28 20:13:55 +00:00
Lunny Xiao
286ede47ad Fix index too many file names bug (#31903)
Try to fix #31884
Fix #28584
2024-09-01 05:57:31 +00:00
Lunny Xiao
c03baab678 Refactor the usage of batch catfile (#31754)
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.

This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
2024-08-20 17:04:57 +00:00
Kemal Zebari
c0b5a843ba Properly filter issue list given no assignees filter (#31522)
Quick fix #31520. This issue is related to #31337.
2024-07-23 18:36:32 +00:00
Carsten Klein
3571b7e3dd Allow searching issues by ID (#31479)
When you are entering a number in the issue search, you likely want the
issue with the given ID (code internal concept: issue index).
As such, when a number is detected, the issue with the corresponding ID
will now be added to the results.

Fixes #4479

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-07-17 00:49:05 +02:00
Lunny Xiao
e4abaff7ff Fix bug filtering issues which have no project (#31337)
Fix #31327
This is a quick patch to fix the bug.
Some parameters are using 0, some are using -1. I think it needs a
refactor to keep consistent. But that will be another PR.
2024-06-14 02:31:07 +00:00
Lunny Xiao
98751108b1 Rename project board -> column to make the UI less confusing (#30170)
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.

But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
2024-05-27 08:59:54 +00:00
wxiaoguang
6f7cd94a02 Fix bleve fuzziness (#30799)
Fix #30797
Fix #30317
2024-05-01 15:32:52 +03:00
silverwind
610802df85 Fix tautological conditions (#30735)
As discovered by https://github.com/go-gitea/gitea/pull/30729.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-30 14:34:40 +02:00
Chongyi Zheng
e80466f734 Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Kemal Zebari
9b7af4340c Perform Newest sort type correctly when sorting issues (#30644)
Should resolve #30642.

Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.

This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
2024-04-23 15:10:01 +08:00
silverwind
74f0c84fa4 Enable more revive linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
wxiaoguang
ca5c895efb Render embedded code preview by permlink in markdown (#30234)
The permlink in markdown will be rendered as a code preview block, like GitHub

Co-authored-by: silverwind <me@silverwind.io>
2024-04-02 17:48:27 +00:00
Lunny Xiao
3f26fe2fa2 Use db.ListOptions directly instead of Paginator interface to make it easier to use and fix performance of /pulls and /issues (#29990)
This PR uses `db.ListOptions` instead of `Paginor` to make the code
simpler.
And it also fixed the performance problem when viewing /pulls or
/issues. Before the counting in fact will also do the search.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 18:51:08 +00:00
wxiaoguang
4734d43e14 Support repo code search without setting up an indexer (#29998)
By using git's ability, end users (especially small instance users) do
not need to enable the indexer, they could also benefit from the code
searching feature.

Fix #29996


![image](https://github.com/go-gitea/gitea/assets/2114189/11b7e458-88a4-480d-b4d7-72ee59406dd1)


![image](https://github.com/go-gitea/gitea/assets/2114189/0fe777d5-c95c-4288-a818-0427680805b6)

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 17:05:00 +01:00
6543
b9c57fb78e Determine fuzziness of bleve indexer by keyword length (#29706)
also bleve did match on fuzzy search and the other way around. this also fix that bug.
2024-03-23 16:45:13 +01:00
Lunny Xiao
f8ab9dafb7 Use db.ListOptionsAll instead of db.ListOptions{ListAll: true} (#29995) 2024-03-22 13:53:52 +01:00
6543
c6e5ec51bd Meilisearch double quote on "match" query (#29740)
make `nonFuzzyWorkaround` unessesary

cc @Kerollmops
2024-03-16 13:19:41 +00:00
6543
1262ff6734 Refactor code_indexer to use an SearchOptions struct for PerformSearch (#29724)
similar to how it's already done for the issue_indexer


---
*Sponsored by Kithara Software GmbH*
2024-03-16 10:32:45 +00:00
6543
7fd0a5b276 Refactor to use optional.Option for issue index search option (#29739)
Signed-off-by: 6543 <6543@obermui.de>
2024-03-13 08:25:53 +00:00