| 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| 7 | 8 | 9 | 10 | 11 | 12 | 13 |
| 14 | 15 | 16 | 17 | 18 | 19 | 20 |
| 21 | 22 | 23 | 24 | 25 | 26 | 27 |
| 28 | 29 | 30 |
- 자바
- kakao
- 컴퓨터과학총론
- techEmpower
- 번역
- C
- 프래그먼트
- 안드로이드
- 개발
- caddy
- server
- unity
- H2O
- it
- android
- 개발자
- C lanuage
- mariadb
- error
- 한글
- java
- NGINX
- php
- 구글
- ubuntu
- 해석
- javascript
- Portfolio
- MySQL
- centOS7
- Today
- Total
개발모음집
TechEmpower Source code & Requirements 한글 번역 본문
TechEmpower
GitHub repository
All of the code used to produce the comparison of web frameworks seen here can be found in the project's GitHub repository. If you have any corrections or contributions, please submit a pull request!
여기서 보여지는 웹 프레임워크의 비교를 위해 사용되는 모든 코드는 프로젝트의 GitHub저장소에서 찾을 수 있습니다.
만약 수정하거나 기역하고 싶다면, pull request를 해주세요.
Forum / mailing list
Join the conversation about this project on the framework-benchmarks Google Group.
이 프로젝트에 대한 대화하기 위해 프레임 워크 - 벤치 마크 Google 그룹 에 가입하세요.
Test requirements
We invite fans of frameworks and especially authors or maintainers of frameworks to join us in expanding the coverage of this project by implementing tests and contributing to the GitHub repository. The following are specifications for each of the test types we have included to-date in this project. Do not be alarmed; the specifications read quite verbose, but that's because they are specifications. The implementations tend to be quite easy in practice.
패스
This project is evolving and we will periodically add new test types. As new test types are added, we encourage but do not require contributors of previous implementations to implement tests for the new test types. Wholly new test implementations are also encouraged to include all test types but are not required to do so. If you have limited time, we recommend you start with the easiest test types (1, 2, 3, and 6) and then continue beyond those as time permits.
패스
General requirements
The following requirements apply to all test types below.
ㅇ
- All test implementations should be production-grade. The particulars of this will vary by framework and platform, but the general sentiment is that the code and configuration should be suitable for a production deployment. The word should is used here because production-grade is our goal, but we don't want this to be a roadblock. If you're submitting a new test and uncertain whether your code is production-grade, submit it anyway and then solicit input from other subject-matter experts.
ㅇ
2. This project intends to measure performance of production-grade deployments of real-world web application frameworks and platforms. All implementations are expected (but not required) to be based on robust implementations of the HTTP protocol. Implementations that are not based on a realistic HTTP implementation will be marked as Stripped (see Questions section for more detail on the Stripped classification). Rather than stipulate a rigid specification of HTTP requirements, we defer to common sense: the implementation should provide a viable subset of HTTP to reasonably unbiased outsiders familiar with HTTP.
ㅇ
3. All test implementations must disable all disk logging. For many reasons, we expect all tests will run without writing logs to disk. Most importantly, the volume of requests is sufficiently high to fill up disks even with only a single line written to disk per request. Please disable all forms of disk logging. We recommend but do not require disabling console logging as well.
ㅇ
4. Specific characters and character case matter. Assume the client consuming your service's JSON responses will be using a case-sensitive language such as JavaScript. In other words, if a test specifies that a map's key is id, use id. Do not use Id or ID. This strictness is required not only because it's sensible but also because our automated validation checks are picky.
ㅇ
5. All test types require Server and Date HTTP response headers. We expect the Server header to be whatever is normal for the platform or framework. If the framework does not normally provide a Server response header, we nevertheless require that you provide one as this roughly normalizes network load across all implementations. For Date, we expect that the rendered date be accurate. However, it does not need to be rendered from the system clock to a byte buffer for each request. Re-rendering once per second is an acceptable tactical optimization (and is an optimization baked into many frameworks).
ㅇ
6. Some form of request routing is required, even if only a single test type (e.g., Plaintext) is implemented. In most cases, the framework's canonical router or a mainstream router library will be used. In some cases, it is considered normal and sufficiently production-grade to use hand-crafted minimalist routing using control structures such as if/else branching. This is acceptable where it is considered canonical for the framework.
ㅇ
Test type 1: JSON serialization
This test exercises the framework fundamentals including keep-alive support, request routing, request header parsing, object instantiation, JSON serialization, response header generation, and request count throughput.
ㅇ
Requirements
- For each request, an object mapping the key
messagetoHello, World!must be instantiated.
ㅇ
2. The recommended URI is /json.
ㅇ
3. A JSON serializer must be used to convert the object to JSON.
ㅇ
4. The response text must be {"message":"Hello, World!"}, but white-space variations are acceptable.
ㅇ
ㅇ
application/json. Note that specifying the character encoding is unnecessary for this content type.ㅇ
Content-Length or Transfer-Encoding.ㅇ
Server and Date.ㅇ
ㅇ
ㅇ
ㅇ
ㅇ
ㅇ
Example request
Example response
ㅇ
Test type 2: Single database query
This test exercises the framework's object-relational mapper (ORM), random number generator, database driver, and database connection pool.
ㅇ
Requirements
- For every request, a single row from a World table must be retrieved from a database table.
ㅇ
2. The recommended URI is /db.
ㅇ
3. The schema for World is id (int, primary key) and randomNumber (int), except for MongoDB, wherein the identity column is _id, with the leading underscore.
ㅇ
4. The World table is known to contain 10,000 rows.
ㅇ
5. The row retrieved must be selected by its id using a random number generator (ids range from 1 to 10,000).
ㅇ
6. The row should be converted to an object using an object-relational mapping (ORM) tool. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
ㅇ
7. The object (or database row, if an ORM is not used) must be serialized to JSON.
ㅇ
8. The response content length should be approximately 32 bytes.
ㅇ
9. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
ㅇ
10. The response headers must include either Content-Length or Transfer-Encoding.
ㅇ
11. The response headers must include Server and Date.
ㅇ
12. Use of an in-memory cache of World objects or rows by the application is not permitted.
ㅇ
13. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
ㅇ
14. gzip compression is not permitted.
ㅇ
15. Server support for HTTP Keep-Alive is strongly encouraged but not required.
ㅇ
16. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
ㅇ
17. The request handler will be exercised at concurrency levels ranging from 8 to 256.
ㅇ
18. The request handler will be exercised using GET requests.
ㅇ
Example request
Example response
Test type 3: Multiple database queries
This test is a variation of Test #2 and also uses the World table. Multiple rows are fetched to more dramatically punish the database driver and connection pool. At the highest queries-per-request tested (20), this test demonstrates all frameworks' convergence toward zero requests-per-second as database activity increases.
ㅇ
Requirements
- For every request, an integer query string parameter named
queriesmust be retrieved from the request. The parameter specifies the number of database queries to execute in preparing the HTTP response (see below).
ㅇ
2. The recommended URI is /queries.
ㅇ
3.The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
ㅇ
4. The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
ㅇ
5. Each row must be selected randomly in the same fashion as the single database query test (Test #2 above).
ㅇ
6. This test is designed to exercise multiple queries, each requiring a round-trip to the database server, and with each resulting row selected individually. It is not acceptable to use batches. It is not acceptable to execute multiple SELECTs within a single statement. It is not acceptable to retrieve all required rows using a SELECT ... WHERE id IN (...) clause.
ㅇ
7. Each World object must be added to a list or array.
ㅇ
8. The list or array must be serialized to JSON and sent as a response.
ㅇ
9.The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
ㅇ
10. The response headers must include either Content-Length or Transfer-Encoding.
ㅇ
11. The response headers must include Server and Date.
ㅇ
12. Use of an in-memory cache of World objects or rows by the application is not permitted.
ㅇ
13. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
ㅇ
14. gzip compression is not permitted.
ㅇ
15. Server support for HTTP Keep-Alive is strongly encouraged but not required.
ㅇ
16. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
ㅇ
17. The request handler will be exercised at 256 concurrency only.
ㅇ
18. The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
ㅇ
19. The request handler will be exercised using GET requests.
ㅇ
Example request
Example response
Test type 4: Fortunes
This test exercises the ORM, database connectivity, dynamic-size collections, sorting, server-side templates, XSS countermeasures, and character encoding.
ㅇ
Requirements
- The recommended URI is /fortunes.
ㅇ
2. A Fortune database table contains a dozen Unix-style fortune-cookie messages.
ㅇ
3. The schema for Fortune is id (int, primary key) and message (varchar), except for MongoDB, wherein the identity column is _id, with the leading underscore.
ㅇ
4. Using an ORM, all Fortune objects must be fetched from the Fortune table, and placed into a list data structure. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
ㅇ
5. The list data structure must be a dynamic-size or equivalent and should not be dimensioned using foreknowledge of the row-count of the database table.
ㅇ
6. Within the scope of the request, a new Fortune object must be constructed and added to the list. This confirms that the data structure is dynamic-sized. The new fortune is not persisted to the database; it is ephemeral for the scope of the request.
ㅇ
7. The new Fortune's message must be "Additional fortune added at request time."
ㅇ
8. The list of Fortune objects must be sorted by the order of the message field. No ORDER BY clause is permitted in the database query (ordering within the query would be of negligible value anyway since a newly instantiated Fortune is added to the list prior to sorting).
ㅇ
9. The sorted list must be provided to a server-side template and rendered to simple HTML (see below for minimum template). The resulting HTML table displays each Fortune's id number and message text.
ㅇ
10. This test does not include external assets (CSS, JavaScript); a later test type will include assets.
ㅇ
11. The HTML generated by the template must be sent as a response.
ㅇ
12. Be aware that the message text fields are stored as UTF-8 and one of the fortune cookie messages is in Japanese.
ㅇ
13. The resulting HTML must be delivered using UTF-8 encoding.
ㅇ
14. The Japanese fortune cookie message must be displayed correctly.
ㅇ
15. Be aware that at least one of the message text fields includes a <script> tag.
ㅇ
16. The server-side template must assume the message text cannot be trusted and must escape the message text properly.
ㅇ
17. The implementation is encouraged to use best practices for templates such as layout inheritence, separate header and footer files, and so on. However, this is not required. We request that implementations do not manage assets (JavaScript, CSS, images). We are deferring asset management until we can craft a more suitable test.
ㅇ
18. The response content type must be set to text/html.
ㅇ
19. The response headers must include either Content-Length or Transfer-Encoding.
ㅇ
20. The response headers must include Server and Date.
ㅇ
21. Use of an in-memory cache of Fortune objects or rows by the application is not permitted.
ㅇ
22. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
ㅇ
23. gzip compression is not permitted.
ㅇ
24. Server support for HTTP Keep-Alive is strongly encouraged but not required.
ㅇ
25. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
ㅇ
26. The request handler will be exercised at concurrency levels ranging from 8 to 256.
ㅇ
27. The request handler will be exercised using GET requests.
ㅇ
Example request
Example response
Minimum template
Along with the example response above, the following Mustache template illustrates the minimum requirements for the server-side template. White-space can be optionally eliminated.
ㅇ
Test type 5: Database updates
This test is a variation of Test #3 that exercises the ORM's persistence of objects and the database driver's performance at running UPDATEstatements or similar. The spirit of this test is to exercise a variable number of read-then-write style database operations.
ㅇ
Requirements
- The recommended URI is /updates.
ㅇ
2. For every request, an integer query string parameter named queries must be retrieved from the request. The parameter specifies the number of rows to fetch and update in preparing the HTTP response (see below).
ㅇ
3. The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
ㅇ
4.The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
ㅇ
5. Each row must be selected randomly using one query in the same fashion as the single database query test (Test #2 above). As with the read-only multiple-query test type (#3 above), use of IN clauses or similar means to consolidate multiple queries into one operation is not permitted. Similarly, use of a batch or multiple SELECTs within a single statement are not permitted.
ㅇ
6. At least the randomNumber field must be read from the database result set.
ㅇ
7. Each World object must have its randomNumber field updated to a new random integer between 1 and 10000.
ㅇ
8. Each World object must be persisted to the database with its new randomNumber value.
ㅇ
9. Use of batch updates is acceptable but not required. To be clear: batches are not permissible for selecting/reading the rows, but batches are acceptable for writing the updates.
ㅇ
10. Use of transactions is acceptable but not required. If transactions are used, a transaction should only encapsulate a single iteration, composed of a single read and single write. Transactions should not be used to consolidate multiple iterations into a single operation.
ㅇ
11. All updates should be completed prior to returning the HTTP response.
ㅇ
12. For raw tests (that is, tests without an ORM), each updated row must receive a unique new randomNumber value. It is not acceptable to change the randomNumber value of all rows to the same random number using an UPDATE ... WHERE id IN (...) clause.
ㅇ
13. Each World object must be added to a list or array.
ㅇ
14. The list or array must be serialized to JSON and sent as a response.
ㅇ
15. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
ㅇ
16. The response headers must include either Content-Length or Transfer-Encoding.
ㅇ
17. The response headers must include Server and Date.
ㅇ
18. Use of an in-memory cache of World objects or rows by the application is not permitted.
ㅇ
19. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
ㅇ
20. gzip compression is not permitted.
ㅇ
21. Server support for HTTP Keep-Alive is strongly encouraged but not required.
ㅇ
22. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
ㅇ
23. The request handler will be exercised at 256 concurrency only.
ㅇ
24. The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
ㅇ
25. The request handler will be exercised using GET requests.
ㅇ
Example request
Example response
ㅇ
Test type 6: Plaintext
ㅇ
This test is an exercise of the request-routing fundamentals only, designed to demonstrate the capacity of high-performance platforms in particular. Requests will be sent using HTTP pipelining. The response payload is still small, meaning good performance is still necessary in order to saturate the gigabit Ethernet of the test environment.
ㅇ
The General Requirements section above applies to all test types, including Plaintext. Importantly, implementations should be production-grade deployments of real-world web application frameworks or platforms. The Plaintext test type is historically highly competitive, leading to some implementations being highly tuned to the express and narrow requirements identified below. We will usually accept such implementations, but they are subject to being classified as Stripped.
ㅇ
Requirements
- The recommended URI is /plaintext.
ㅇ
2. The response content type must be set to text/plain.
ㅇ
3. The response body must be Hello, World!.
ㅇ
4. This test is not intended to exercise the allocation of memory or instantiation of objects. Therefore it is acceptable but not required to re-use a single buffer for the response text (Hello, World). However, the response must be fully composed from the response text and response headers within the scope of each request and it is not acceptable to store the entire payload of the response, or an unnaturally large subset of the response, headers inclusive, as a pre-rendered buffer. "Buffer" here refers to a byte array, byte buffer, character array, character buffer, string, or string-like data structure. The spirit of the test is to require the construction of the HTTP response as is typically done by a framework or platform via concatenation of strings or similar. For example, pre-rendering a buffer with HTTP/1.1 200 OK<cr>Content-length: 15<cr>Server: Example<cr> would not be acceptable.
ㅇ
5. The response headers must include either Content-Length or Transfer-Encoding.
ㅇ
6. The response headers must include Server and Date.
ㅇ
7. gzip compression is not permitted.
ㅇ
8. Server support for HTTP Keep-Alive is strongly encouraged but not required.
ㅇ
9. Server support for HTTP/1.1 pipelining is assumed. Servers that do not support pipelining may be included but
ㅇ
10. should downgrade gracefully. If you are unsure about your server's behavior with pipelining, test with the wrk load generation tool used in our tests.
ㅇ
11. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
ㅇ
12. The request handler will be exercised at 256, 1024, 4096, and 16,384 concurrency.
ㅇ
13. The request handler will be exercised using GET requests.
ㅇ
Example request
Example response
'Benchmark' 카테고리의 다른 글
| vmware 가상서버 포트포워딩 (0) | 2017.10.15 |
|---|---|
| Jmeter 사용법 (0) | 2017.10.06 |
| TechEmpower Benchmarks Document 한글 번역 (0) | 2017.09.21 |
| TechEmpower Environment 한글 번역 (0) | 2017.09.20 |
| TechEmpower Motivation & Questions 한글 번역 (0) | 2017.09.19 |
